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FOREWORD 


This  work  was  conducted  by  Search  Technology,  Inc.  under  the  Augmented 
Maintenance  Training  Project,  On-site  Training,  Work  Unit  RF63-522-801-002-03.44  and 
sponsored  by  the  Office  of  Naval  Research. 

This  paper  defines  the  current  state  of  research  on  the  subject  of  fault  isolation 
training  of  military  technicians  and  establishes  a  benchmark  for  continued  research.  It 
was  originally  published  in  the  Journal  of  the  Human  Factors  Society,  Vol.  27,  No.  5,  and 
is  being  reprinted  by  Navy  Personnel  Research  and  Development  Center  for  wider 
distribution. 

The  contracting  officer's  technical  representative  was  George  Lahey. 


H.  S.  ELDREDGE 
Commander,  U.S.  Navy 
Commanding  Officer 
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SUMMARY 


Background 

Data  from  the  field  and  from  the  formal  schools  indicate  that  troubleshooting  skills 
do  not  transfer  from  one  system  to  another  and  that  retraining  becomes  necessary 
whenever  technicians  are  assigned  to  maintain  an  unfamiliar  system.  Detaching  personnel 
from  their  units  to  send  them  back  to  school  results  in  loss  of  operational  readiness  and 
excessive  costs.  A  way  to  avoid  both  the  excessive  costs  and  the  loss  of  operational 
readiness  is  needed 

Purpose 

This  effort  was  conducted  to  review  the  literature  on  troubleshooting  training,  assess 
the  effectiveness  of  current  methods,  and  establish  a  basis  for  further  research. 

Approach 

This  literature  search  focused  on  data  regarding  the  individual  skills  needed  for 
successful  fault  isolation  with  emphasis  on  the  training  issues  involved.  Four  sources  were 
consulted:  PSYCHINFO  (the  online  data  base  from  the  Psychological  Abstracts),  the  NTIS 
data  base,  the  personal  files  of  the  authors,  and  personal  and  professional  contacts. 
Literature  that  did  not  specifically  involve  training  issues,  including  the  use  of  job  aids, 
was  not  reviewed.  About  80  reports  and  articles  relevant  to  troubleshooting,  problem 
solving  with  test  equipment,  problem  solving  in  technical  schools  and  that  presented 
actual  performance  data  were  examined. 

Results  and  Conclusions 


The  successful  troubleshooter  must  be  able  to  (1)  repair  and  replace  components,  (2) 
conduct  tests,  and  (3)  select  an  appropriate  strategy.  The  last  area  seems  to  pose  the 
greater  challenge  to  instructional  designers  seeking  to  facilitate  transfer  of  training.  The 
themes  arising  in  the  literature  with  regard  to  selection  of  strategies  are  that  (1)  humans 
are  not  good  at  judging  probabilities,  (2)  performance  degrades  as  systems  become  larger 
and  more  complex,  (3)  performance  degrades  also  in  the  face  of  time  constraints,  (4) 
presentation  of  theory  of  operation  generally  does  not  improve  performance,  and  (5) 
proceduralization  improves  performance.  Because  recent  research  does  not  address  the 
problems  of  transfer  of  training,  the  utility  of  high-technology  troubleshooting  training 
techniques  in  the  real  world  has  not  been  demonstrated. 

Recommendations 


Troubleshooting  research  studies  need  to  concentrate  on  ways  to  proceduralize  the 
troubleshooting  task,  instill  the  use  of  effective  algorithms  and  heuristics,  and  provide 
effective  practice  in  logical  troubleshooting.  The  prerequisites  for  effective  transfer  of 
training  to  the  actual  equipment  also  need  close  attention. 
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Review  and  Evaluation  of  Empirical  Research 
in  Troubleshooting 

NANCY  M.  MORRIS  and  WILLIAM  B.  ROUSE,  Search  Technology,  Inc.,  Norcross,  Georgia 


Following  an  analysis  of  task  requirements  for  successful  troubleshooting,  this  paper  con¬ 
siders  human  abilities,  limitations,  and  inclinations  with  respect  to  troubleshooting.  Re¬ 
search  on  the  effects  of  various  approaches  to  the  training  of  troubleshooting  is  reviewed. 
The  extent  to  which  troubleshooting  performance  is  influenced  by  instruction  is  highly  re¬ 
lated  to  the  level  of  explicitness  of  action-related  information  provided.  An  approach  that 
forces  people  to  use  their  system  knowledge  explicitly  is  a  promising  alternative  to  explicit 
instruction  in  algorithms  or  diagnostic  heuristics,  but  such  an  approach  is  not  supported  by 
data  from  transfer  studies.  A  combination  of  the  two  approaches  may  be  the  most  effective 
means  of  teaching  troubleshooting,  and  research  evaluating  the  soundness  of  this  idea 
should  be  conducted. 


INTRODUCTION 

The  task  of  troubleshooting  may  be  de¬ 
scribed  rather  simply.  Given  that  a  system  is 
not  functioning  properly,  the  troubleshooter 
must  attempt  to  locate  the  reason  for  the 
malfunction  and  must  then  repair  or  replace 
the  faulty  component.  The  level  of  specificity 
with  which  the  source  of  the  problem  is  iden¬ 
tified  depends  on  the  troubleshooter’s  role 
and  the  current  demand  characteristics  of 
the  situation.  For  example,  the  trouble¬ 
shooter  may  elect  to  replace  a  component  or 
module,  or  perform  some  compensatory  ac¬ 
tion  to  enable  the  system  to  continue  func¬ 
tioning  temporarily. 

What  skills  are  required  to  troubleshoot? 
Insight  into  the  abilities  necessary  may  be 
gained  by  considering  what  the  trouble¬ 


shooter  is  expected  to  do.  If  repair  of  the 
system  is  expected,  then  the  troubleshooter 
must  be  able  to  repair  or  replace  system  com¬ 
ponents.  Depending  on  the  actions  required 
to  do  so,  various  abilities  may  be  needed.  For 
example,  the  troubleshooter  may  be  expected 
to  identify  components,  use  hand  tools  ap¬ 
propriately,  and/or  perform  relatively  simple 
actions,  such  as  soldering  a  connection.  The 
ability  to  replace/repair  system  components 
is  largely  system-specific,  particularly  if 
components  must  be  identified.  Skills  re¬ 
quired  are  generic  to  the  extent  that  the 
systems  encountered  by  the  troubleshooter 
contain  similar  components  and  require  the 
use  of  the  same  tools. 

It  seems  possible,  at  least  in  principle,  that 
someone  having  only  the  ability  to  replace 
parts  could  successfully  troubleshoot  a 
system.  The  troubleshooter  could  simply  re¬ 
place  parts  until  the  system  was  repaired.  Of 
course,  if  the  malfunction  involved  adjust- 
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ment  problems  rather  than  defective  compo¬ 
nents,  this  approach  would  not  be  successful. 
In  addition  to  these  limiting  conditions,  a  re- 
place-until-fixed  approach  would  often  be 
undesirable  because  of  the  accompanying  ex¬ 
pense  of  repairing  the  system;  a  number  of 
unnecessary  replacements  would  be  expected 
to  occur  if  such  an  approach  were  used. 
Therefore,  the  ability  to  make  tests  in  order  to 
eliminate  components  from  consideration  is 
desirable  in  a  troubleshooter. 

There  are  several  important  aspects  of 
making  tests.  The  troubleshooter  should  be 
able  to  identify  the  tests  that  would  supply 
the  information  desired,  identify  and  access 
test  points,  select  and  use  test  equipment  ap¬ 
propriately,  and  make  inferences  as  to  the  re¬ 
sults  of  each  test  (e.g.,  the  component  is  good 
or  bad,  or  the  reading  is  abnormally  high  or 
low).  As  with  the  ability  to  replace  or  repair 
components,  the  ability  to  make  tests  is 
system-specific  in  many  respects  (e.g.,  the 
ability  to  identify  test  points  and  make  infer¬ 
ences  as  to  whether  a  given  reading  is  off  or 
normal).  Abilities  required  to  make  tests  are 
generic  to  those  systems  in  which  similar 
tests  may  be  made.  For  example,  the  use  of 
an  oscilloscope  may  be  very  useful  in  a  va¬ 
riety  of  electronics  troubleshooting  problems 
but  may  not  be  helpful  at  all  in  diagnosing 
problems  in  a  chemical  plant. 

There  is  another  "ability”  that  seems  to  be 
helpful  in  troubleshooting.  This  is  the  ability 
to  search  for  the  problem  in  a  systematic 
manner — in  short,  to  employ  some  kind  of 
strategy  in  searching  for  the  source  of  the  dif¬ 
ficulty.  Depending  on  the  strategy  selected, 
other  abilities  are  required.  Troubleshooting 
strategies  may  take  many  forms,  from  simply 
starting  with  the  component  nearest  to  the 
troubleshooter  and  tracing  back  to  the 
source  of  the  problem,  to  generating  hypoth¬ 
eses  based  upon  knowledge  of  the  system  and 
symptoms  and  identifying  tests  to  confirm  or 
reject  those  hypotheses.  Troubleshooters 
may  also  employ  heuristics  (i.e.,  rules  of 


thumb  that  are  useful  for  solving  a  problem), 
adopt  a  half-split  or  bracketing  approach,  or 
follow  a  predetermined  set  of  procedures.  In¬ 
tuitively,  it  seems  that  troubleshooting  strat¬ 
egies  have  more  potential  generality  than 
does  knowledge  of  how  to  make  tests  or  re¬ 
place  parts,  although  troubleshooting  strate¬ 
gies  may  be  very  system-specific  (e.g.,  testing 
a  particular  component  first  because  it  has  a 
high  failure  rate). 

Good  versus  Poor  Troubleshooters 

The  importance  of  these  three  general  abil¬ 
ities  to  successful  troubleshooting  is  illus¬ 
trated  by  the  results  of  studies  in  which  effec¬ 
tive  and  ineffective  troubleshooters  have 
been  compared.  For  example,  Saupe  (1954) 
observed  technicians  repairing  radio  re¬ 
ceivers  and  noted  that  poor  troubleshooters 
had  more  incorrect  hypotheses  and  pursued 
incorrect  hypotheses  longer  than  did  good 
troubleshooters.  They  were  also  less  likely  to 
recognize  critical  information  as  such  and 
tended  to  make  fewer  checks  before  ac¬ 
cepting  a  hypothesis  as  correct.  There  were 
no  apparent  differences  between  good  and 
poor  troubleshooters  in  their  ability  to  per¬ 
ceive  symptoms  completely  and  correctly. 
Both  groups  made  approximately  the  same 
number  of  errors  using  test  equipment  and 
they  duplicated  checks  equally  often. 

Saltz  and  Moore  (1953)  observed  techni¬ 
cians  who  had  been  classified  by  their  super¬ 
visors  as  either  good  or  poor  troubleshooters, 
and  they  noted  several  strategic  hindrances 
to  effective  troubleshooting.  Ineffective  trou¬ 
bleshooters  tended  to  avoid  difficult  checks, 
made  difficult  checks  when  simpler  checks 
would  have  been  sufficient,  made  repeat 
checks  needlessly,  made  irrelevant  checks, 
and  omitted  relevant  checks.  Saltz  and 
Moore  also  observed  that  good  trouble¬ 
shooters  knew  more  about  the  functioning  of 
the  equipment  and  were  better  able  to  use 
test  equipment  properly. 

Highland,  Newman,  and  Waller  (1956)  re- 
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ported  that  successful  and  unsuccessful  trou¬ 
bleshooters  required  approximately  the  same 
amount  of  time  to  solution  and  made  the 
same  number  of  tests,  but  that  they  differed 
in  the  tests  they  made.  Good  troubleshooters 
made  fewer  repetitive  checks  involving  varia¬ 
tion  of  controls,  and  more  repetitive  checks  of 
other  types.  They  were  also  more  likely  than 
were  poor  troubleshooters  to  use  schematics 
on  the  problems  they  missed,  and  they  gener¬ 
ally  appeared  to  use  a  more  varied  attack  in 
solving  problems. 

Baldwin  (1978)  summarized  research  ana¬ 
lyzing  radar  mechanics’  weaknesses,  and  re¬ 
ported  a  number  of  differences  between  ef¬ 
fective  and  ineffective  mechanics.  For  ex¬ 
ample,  ineffective  mechanics  made  fewer 
checks  involving  manipulation  of  control  set¬ 
tings  and  generally  failed  to  observe  all  gross 
symptoms.  In  contrast,  they  used  schematics 
earlier  and  more  frequently  than  did  effective 
mechanics,  and  they  made  more  circuitry 
checks  and  measurements  involving  test 
equipment.  After  localizing  a  problem,  inef¬ 
fective  mechanics  spent  more  time  before  at¬ 
tempting  replacement!  required  a  longer 
time  to  find  physical  locations  of  compo¬ 
nents,  and  made  more  errors  in  repair  or  re¬ 
placement  of  components.  Once  repair  was 
complete,  ineffective  mechanics  did  less 
checking  to  verify  repair  and  failed  more 
often  to  return  the  system  to  operational 
status. 

Similar  observations  have  been  noted  by 
Glaser  and  Phillips  (1954),  Moore,  Saltz,  apd 
Hoehn  (1955),  and  McDonald,  Waldrop,  and 
White  (1983).  As  an  interesting  note,  Glaser 
and  Phillips  reported  that  although  failure  to 
solve  problems  was  associated  with  strategic 
shortcomings,  such  as  insufficient  relevant 
tests,  more  than  20%  of  all  such  failures 
could  be  attributed  to  faulty  inferences  (i.e., 
misinterpretation  of  an  appropriate  test). 

Summary.  It  is  possible  to  summarize  the 
characteristics  of  poor  troubleshooters  rela¬ 
tive  to  the  three  general  abilities  cited  as 


being  important  for  successful  trouble¬ 
shooting.  With  respect  to  repair  and  replace¬ 
ment  of  components,  ineffective  trouble¬ 
shooters  demonstrated  a  lack  of  elementary 
knowledge  and  were  poor  in  executing  and 
verifying  the  results  of  their  work.  When  per¬ 
forming  tests,  poor  troubleshooters  made 
fewer  useful  tests  and  more  useless  tests  and 
were  inconsistent  in  their  consideration  of 
test  difficulty.  The  strategic  behavior  of  poor 
troubleshooters  was  characterized  by  incom¬ 
plete  and  inappropriate  use  of  information, 
ineffective  hypothesis  generation  and  testing, 
and  generally  less  strategic  flexibility. 

Training  Requirements 

If  we  accept  the  necessity  of  being  able  to 
repair  or  replace  components,  perform  tests, 
and  employ  a  strategy,  then  those  respon¬ 
sible  for  training  troubleshooters  must  pro¬ 
vide  training  in  three  broad  areas.  Strategic 
training  would  seem  to  pose  the  greatest  con¬ 
ceptual  challenge  to  the  instructor.  Repair 
and  replacement  training  seems  relatively 
straightforward;  trainees  must  be  able  to  re¬ 
pair  or  replace  those  components  that  re¬ 
quire  it.  Training  persons  to  perform  tests 
seems  more  complicated,  particularly  if  the 
results  of  several  tests  must  be  assimilated  in 
order  to  make  inferences,  but  such  training  is 
at  least  constrained  by  the  number  of  pos¬ 
sible  tests. 

By  comparison,  one  has  a  great  deal  of  lati¬ 
tude  in  approaching  strategic  trouble¬ 
shooting  training.  In  order  to  proceed,  one 
must  answer  a  number  of  questions.  For  ex¬ 
ample,  should  people  be  trained  to  employ  a 
particular  strategy  or  should  they  be  allowed 
to  discover  their  own  strategies  within  some 
envelope  of  acceptability?  If  the  use  of  a  par¬ 
ticular  strategy  is  desired,  can  people  be 
taught  to  use  it?  What  is  the  best  way  to 
teach  a  given  strategy?  And  finally,  what 
strategy  should  be  taught? 

The  answer  to  the  last  question  is  compli¬ 
cated  by  the  fact  that  there  is  no  single  ap- 
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proach  that  is  appropriate  to  all  situations. 
The  best  strategy  for  a  given  situation  de¬ 
pends  on  the  degree  to  which  use  of  a  given 
strategy  can  enable  the  troubleshooter  to 
meet  current  performance  criteria  and  the  fa¬ 
cility  and  willingness  with  which  the  human 
troubleshooter  may  be  expected  to  employ 
the  approach. 

With  respect  to  performance  criteria,  it  is 
possible  to  imagine  four  goals:  speed,  accu¬ 
racy,  low  cost,  and  efficiency.  The  goal  of  effi¬ 
ciency  is  probably  most  appropriate  as  a 
subgoal  for  speed  and/or  cost,  since  mini¬ 
mizing  the  number  of  tests  may  in  turn  be  a 
way  of  achieving  these  goals.  Obviously  the 
criteria  noted  here  are  not  orthogonal,  and 
all  seem  desirable.  At  any  given  time,  how¬ 
ever,  trade-offs  may  be  made  as  to  which  cri¬ 
terion  is  dominant,  and  this  may  determine 
which  troubleshooting  approach  should  be 
adopted.  For  example,  if  time  is  of  the  es¬ 
sence  or  the  risks  associated  with  inaccuracy 
are  high,  saving  the  cost  of  a  spare  part 
should  not  be  uppermost  in  the  trouble¬ 
shooter's  mind.  On  the  other  hand,  if  the  pace 
is  more  leisurely,  the  troubleshooter  may  be 
more  concerned  about  the  cost  of  repair. 

HUMAN  ABILITIES,  LIMITATIONS, 
AND  INCLINATIONS 

The  troubleshooter’s  ability  to  employ  a 
given  strategy  is  related  to  human  informa¬ 
tion-processing  abilities  and  limitations.  It  is 
important  that  the  human’s  abilities  be  con¬ 
sidered  because  of  the  errors  that  may  occur 
as  a  result  of  a  mismatch  between  abilities 
required  and  abilities  available.  For  ex¬ 
ample,  suppose  it  is  determined  that  a  half¬ 
split  approach,  which  allows  maximum  in¬ 
formation  gain  per  test,  is  theoretically  the 
best  approach  for  a  given  problem.  If  the 
troubleshooter  has  difficulty  in  identifying 
the  set  of  remaining  alternatives  following  a 
test,  then  errors  could  result  in  a  loss  of  effi¬ 
ciency  and  incorrect  inferences.  Insight  into 


characteristics  of  human  information-pro¬ 
cessing  that  may  affect  the  troubleshooting 
process  may  be  gained  from  studies  in  which 
people  were  observed  as  they  solved 
problems. 

For  the  most  part,  the  studies  reviewed  in 
this  section  involved  the  solution  of  relatively 
simple,  low-fidelity  troubleshooting 
problems.  Although  a  variety  of  media  were 
employed  (paper-and-pencil  tasks,  computer- 
based  simulations,  and  in  a  few  cases,  actual 
hardware),  a  characteristic  common  to  many 
of  these  problems  is  that  they  were  context- 
free,  or  generic.  In  other  words,  they  were  not 
intended  to  be  representative  of  any  partic¬ 
ular  system.  For  example,  in  the  case  of  a 
paper-and-pencil  or  computer-generated 
task,  "components”  might  simply  be  unla¬ 
beled  rectangles,  with  lines  between  rec¬ 
tangles  representing  functional  connections. 
Typically,  subjects  attempted  to  locate  faulty 
components  by  requesting  information  about 
the  quality  of  component  inputs  and/or 
outputs. 

Effects  of  Visual  Characteristics 

One  insight  gained  from  these  studies  is 
that  performance  may  be  affected  by  the  vi¬ 
sual  characteristics  of  the  problem.  Dale 
(1958)  presented  college  students  with  a 
simple  paper-and-pencil  troubleshooting 
task  and  found  that  initial  tests  tended  to  be 
in  the  middle  of  the  picture,  without  regard 
to  functional  relationships  between  compo¬ 
nents.  Students  also  made  frequent  tests  at 
bends  in  component  chains,  even  though 
such  features  were  irrelevant  to  component 
outputs.  Dale  also  noted  that  his  students 
tended  to  adopt  a  pattern-like  strategy  when 
searching  for  a  faulty  component  among 
unrelated  components;  however,  no  such 
strategy  was  observed  when  students  were 
not  shown  a  picture  but  performed  the  analo¬ 
gous  task  of  guessing  which  letter  of  the  al¬ 
phabet  was  being  thought  of. 
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Brooke  and  Duncan  (1981)  studied  the 
troubleshooting  behavior  of  college  students 
as  they  solved  computer-based  problems. 
Connections  between  components  were  de¬ 
picted  as  straight  diagonal  lines  or  as 
crooked  lines  composed  of  vertical  and  hori¬ 
zontal  segments,  and  "flow"  through  connec¬ 
tions  could  be  either  left-to-right  or  uncon¬ 
strained.  Troubleshooting  was  faster  and 
fewer  diagnostic  errors  were  made  when  con¬ 
nections  were  straight  lines  and  flow  was 
left-to-right  only. 

Rigney  and  Hoffman  (1961)  noted  that  vi¬ 
sual  clutter  had  no  effect  upon  subjects' 
ability  to  solve  paper-and-pencil  trouble¬ 
shooting  problems.  However,  Elliott  (1965) 
reported  differences  in  high  school  students' 
performance  on  paper-and-pencil  problems 
related  to  the  format  of  the  troubleshooting 
aid  (i.e.,  procedures)  provided.  Accuracy  was 
greater  when  subjects  used  a  block  diagram 
rather  than  a  list  of  actions  to  take.  Problems 
were  also  solved  more  quickly  and  accurately 
when  the  level  of  detail  of  the  aid  was  no 
greater  than  necessary  to  solve  the  problems. 

Although  all  of  these  studies  used  simple 
simulated  tasks,  the  effects  that  irrelevant 
problem  characteristics  may  have  upon 
troubleshooting  performance  are  not  limited 
to  the  laboratory.  Wohl  (1961)  noted  that  dif¬ 
ferences  in  system  down  time  were  asso¬ 
ciated  with  differences  in  "packaging  tech¬ 
niques." 

Effects  of  System  Complexity  and 
Time  Constraints 

Troubleshooting  performance  may  be  de¬ 
graded  by  the  size  and  complexity  of  the 
system.  Brooke  and  Duncan  (1981)  noted  that 
increasing  the  number  of  components  in 
their  generic  troubleshooting  task  resulted  in 
more  diagnostic  errors  made  by  subjects. 
Rouse  (1978a,  1979b,  1979c),  using  a  task 
similar  to  that  employed  by  Brooke  and 
Duncan,  reported  that  subjects  departed 


from  optimal  (i.e.,  did  not  achieve  the  max¬ 
imum  reduction  of  the  number  of  alterna¬ 
tives  with  each  test)  and  took  longer  to  solve 
problems  as  network  size  increased. 

Goldbeck,  Bernstein,  Hillix,  and  Marx 
(1957),  using  a  paper-and-pencil  task,  found 
that  more  tests  were  made  as  the  number  of 
relevant  relationships  between  component 
chains  increased.  Rouse  and  Rouse  (1979)  re¬ 
ported  that  the  amount  of  time  subjects  re¬ 
quired  to  solve  context-free  problems  was 
highly  correlated  with  the  number  of  rele¬ 
vant  functional  relationships  that  must  be 
considered.  McDonald  et  al.  (1983)  noted 
similar  increases  in  time  required  and 
number  of  probes  made  accompanying  in¬ 
creases  in  the  complexity  of  actual  elec¬ 
tronics  boards. 

As  with  visual  characteristics,  the  effects  of 
system  complexity  are  apparently  similar 
across  a  variety  of  systems.  Wohl  (1982)  ex¬ 
amined  available  data  on  time  required  to 
repair  various  systems  and  devised  a  model 
to  predict  system  repair  time.  One  of  the  pa¬ 
rameters  of  that  model  is  a  complexity  index, 
based  upon  the  number  of  relevant  relation¬ 
ships  between  components. 

Feedback  loops  also  appear  to  present 
problems  for  troubleshooters.  Rouse  (1979a) 
observed  the  troubleshooting  behavior  of 
four  students  as  they  attempted  to  locate 
faults  in  a  simple  network  of  components 
containing  feedback  loops.  A  model  was  then 
developed  to  match  their  behavior.  One  im¬ 
portant  parameter  of  the  model,  which  ap¬ 
peared  useful  in  distinguishing  strategies, 
was  the  degree  to  which  information  in  feed¬ 
back  loops  was  used  Two  subjects  seemed  to 
ignore  the  feedback  loops  and  made  more  in¬ 
correct  diagnoses  than  did  the  other  two.  Al¬ 
though  the  results  of  this  study  are  not  strong 
enough  to  warrant  drawing  definitive  con¬ 
clusions,  the  notion  that  feedback  poses 
problems  for  troubleshooters  is  one  with 
which  many  instructors  will  agree. 
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As  a  final  characteristic,  it  is  possible  to 
note  that  humans'  strategies  may  be  affected 
by  time  constraints.  Rouse  (1978a)  varied  the 
amount  of  time  in  which  subjects  were  al¬ 
lowed  to  solve  simulated  troubleshooting 
problems.  As  less  time  was  allowed,  the 
number  of  tests  made  by  subjects  departed 
from  optimal.  This  effect  was  observed  even 
when  the  amount  of  time  allowed  was 
greater  than  the  average  amount  of  time  that 
self-paced  subjects  had  previously  required 
to  solve  the  problems. 

Generation  of  Hypotheses 

Many  approaches  to  troubleshooting  re¬ 
quire  the  troubleshooter  to  hypothesize 
about  the  possible  causes  of  symptoms.  There 
is  some  evidence  to  suggest  that  people  may 
have  difficulty  in  generating  complete  sets  of 
hypotheses.  Mehle  (1980)  presented  experi¬ 
enced  and  novice  mechanics  with  scenarios 
describing  automobile  problems  and  asked 
them  to  identify  all  of  the  potential  causes  of 
these  problems.  While  doing  so,  subjects  gen¬ 
erated  verbal  protocols,  which  were  re¬ 
corded.  Their  lists  of  hypotheses  were  gener¬ 
ally  incomplete;  yet,  subjects  were  frequently 
overconfident  about  the  completeness  of 
their  lists.  Gettys,  Manning,  Mehle,  and 
Fisher  (1980)  reviewed  a  series  of  related  ex¬ 
periments  using  a  variety  of  problems  (in¬ 
cluding  Mehle’s  study)  and  reported  similar 
results.  They  also  observed  that  subjects 
seemed  to  check  hypotheses  for  logical  con¬ 
sistency  with  available  information  .as  they 
were  generated. 

Use  of  Negative  Information 

There  is  some  evidence  to  indicate  that 
during  the  course  of  troubleshooting,  people 
use  positive  information  (i.e.,  information 
about  bad  outputs)  but  do  not  take  account  of 
negative  information  (i.e.,  information  about 
what  has  not  failed).  Rouse  (1978b)  presented 
a  fuzzy-set  model  of  subjects’  performance  in 


solving  simulated  troubleshooting  tasks.  Al¬ 
though  parameters  for  inclusion  in  both  fea¬ 
sible  and  infeasible  sets  (i.e.,  components 
that  could  and  could  not  be  the  failure)  were 
included,  only  the  parameter  related  to  the 
criterion  for  inclusion  in  the  infeasible  set 
was  sensitive  to  differences  in  performance. 

Use  of  Probabilistic  Information 

If  the  sampling  behavior  of  troubleshooters 
may  be  used  as  an  indication,  then  people 
cannot  be  viewed  as  purely  Bayesian  when 
evaluating  hypotheses.  In  an  experiment  con¬ 
ducted  by  Bond  and  Rigney  (1966),  Navy 
technicians  were  asked  to  judge  the  proba¬ 
bilities  of  occurrence  of  various  symptoms  if 
certain  malfunctions  should  occur.  The  re¬ 
sulting  fault-symptom  matrices  were  then 
used  in  conjunction  with  Bayes'  theorem  to 
predict  the  sequence  of  checks  that  would  be 
made  when  performing  a  simulated  trouble¬ 
shooting  task.  Predictions  were  in  agreement 
with  actual  behavior  approximately  50%  of 
the  time  It  was  also  noted  that  the  degree  to 
which  troubleshooting  performance’ was 
Bayesian  was  moderately  related  to  the  accu¬ 
racy  of  the  initial  fault-symptom  matrix  gen¬ 
erated. 

Troubleshooters  apparently  experience 
some  difficulty  in  judging  the  likelihood  that 
various  components  will  fail.  Mills  (1971) 
presented  subjects  with  a  simple  chain  of 
components  and  asked  them  to  locate  the 
faulty  one.  Over  the  course  of  the  experiment, 
he  employed  different  distributions  to  deter¬ 
mine  the  probability  of  failure  for  each  com¬ 
ponent.  Search  strategies  employed  by  sub¬ 
jects  appeared  to  be  ‘‘statistically  logical” 
but  were  not  optimal;  over  time,  subjects  ap¬ 
peared  to  take  the  current  failure  proba¬ 
bilities  into  account,  except  when  the  most 
likely  failure  was  at  either  end  of  the  chain. 

Related  information  is  available  from  Sto- 
lurow,  Bergum,  Hodgson,  and  Silva  (1955). 
Prior  to  implementing  a  proposed  approach 
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to  strategic  training,  it  was  necessary  to  ask 
instructors  to  judge  the  probabilities  with 
which  components  in  aircraft  power  plants 
would  fail.  The  approach  was  not  imple¬ 
mented  because  sufficient  agreement  be¬ 
tween  instructors  could  not  be  reached. 

Individual  Differences 

A  number  of  the  studies  reviewed  reported 
that  performance  was  substantially  related 
to  individual  differences,  which  often  re¬ 
sulted  in  interactions  with  training  tech¬ 
niques.  The  influences  of  three  variables  have 
been  systematically  studied.  These  are  the 
variables  of  experience,  relevant  abilities, 
and  cognitive  style. 

Experience.  It  is  not  surprising  that  differ¬ 
ences  in  performance  related  to  experience 
have  been  observed.  After  all,  it  has  often 
been  said  that  practice  makes  perfect.  The 
role  of  experience  is  discussed  here  because 
some  of  the  research  reports  present  findings 
that  are  not  intuitively  obvious. 

Vineberg  (1955/1968)  compared  the  abili¬ 
ties  of  field-experienced  mechanics  and  re¬ 
cent  training  school  graduates  to  repair  the 
AAFCS  M33  radar.  As  might  be  expected,  the 
experienced  personnel  scored  much  higher 
than  did  the  novices  on  a  performance  test  in 
which  speed  of  repair  was  emphasized.  An  in¬ 
teresting  feature  of  this  report  was  a  discus¬ 
sion  of  the  relative  speed  with  which  various 
performance  components  were  acquired. 
"Energizing  and  operation"  skills  were  ac¬ 
quired  very  early,  being  demonstrated  by 
mechanics  of  all  experience  levels.  "Field  ad¬ 
justment  and  preventive  maintenance"  was 
acquired  somewhat  later,  becoming  rela¬ 
tively  stable  after  1  to  6  months  of  experi¬ 
ence.  "Troubleshooting"  developed  over  a 
long  period  of  time,  continuing  to  improve 
through  the  highest  reported  experience  level 
of  25  to  48  months.  Two  interpretations  of 
these  differences  are  possible.  One  is  that 
these  performance  components  are  acquired 


at  different  rates  and  thus  require  different 
amounts  of  practice;  the  other  is  that  daily 
experience  afforded  these  mechanics  less  op¬ 
portunity  for  troubleshooting  practice,  so 
that  various  skills  were  acquired  at  different 
times. 

Apparently  the  ability  to  learn  from  experi¬ 
ence  also  improves  with  experience.  Rouse 
(1979b,  1979c)  reported  that  students  in  the 
last  semester  of  an  aircraft  maintenance 
training  course  showed  positive  transfer 
from  an  aided  to  an  unaided  version  of  a 
simple,  context-free  troubleshooting  task. 
(Aiding  consisted  of  identifying  those  alter¬ 
natives  that  had  been  logically  eliminated  by 
tests.)  No  such  effect  was  found  for  first-se¬ 
mester  trainees.  Note  that  the  experience 
variable  in  this  research  was  general  trouble¬ 
shooting  experience  within  a  domain  and  not 
amount  of  practice  on  the  experimental  task. 

Hunt  and  Rouse  (1981)  again  noticed  dif¬ 
ferences  in  first-  and  last-semester  trainees  in 
transfer  to  a  context-specific  troubleshooting 
task  (which  is  described  in  a  later  section  of 
this  paper).  Last-semester  trainees  obtained 
more  information  per  test  in  the  context-spe¬ 
cific  problems  after  training  with  the  aided 
context-free  task.  First-semester  trainees 
showed  negative  transfer,  obtaining  less  in¬ 
formation  per  test.  Hunt  and  Rouse  attrib¬ 
uted  these  differences  to  probable  differences 
in  the  way  the  aiding  was  perceived.  The 
more  experienced  subjects  viewed  the  aid  as 
helping  them,  and  they  were  able  to  deter¬ 
mine  the  principles  used  by  the  aid  and  em¬ 
ploy  them  in  an  unaided  situation.  The  less 
experienced  subjects  simply  viewed  the 
aiding  as  a  way  to  make  the  task  easier  for 
them,  and  did  not  learn  the  aiding  principles. 

Although  experience  has  been  shown  to  be 
an  important  factor  affecting  trouble¬ 
shooting  performance,  differences  in  perfor¬ 
mance  as  a  result  of  experience  are  less  ap¬ 
parent  as  more  troubleshooting  guidance  is 
provided.  Potter  and  Thomas  (1976)  found 
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that  maintenance  technicians  with  experi¬ 
ence  of  six  months  or  less  solved  fewer 
problems  and  used  more  spare  parts  than  did 
more  experienced  personnel.  However,  these 
differences  were  observed  only  when  the 
problems  were  accompanied  by  standard 
technical  orders.  When  troubleshooting  pro¬ 
cedures  were  available,  no  experience-re¬ 
lated  differences  were  observed.  Similar  re¬ 
sults  were  noted  in  a  study  by  Elliott  and 
Joyce  (1971)  in  which  high  school  students 
using  a  troubleshooting  guide  were  able  to 
identify  faults  in  electronic  equipment  as  ef¬ 
fectively  as  Air  Force  technicians  using  tradi¬ 
tional  manuals. 

Abilities  and  aptitudes.  Duncan  (1971)  noted 
a  difference  between  subjects  with  high 
ability  and  low  ability  in  their  retention  of 
skills  learned  via  troubleshooting  a  paper- 
and-pencil  simulation  with  a  decision  tree. 
Subjects  were  classified  as  having  high  or 
low  ability  on  the  basis  of  their  pretraining 
performance  of  the  task.  Over  a  period  of  182 
days  following  the  removal  of  the  decision 
tree,  performance  of  the  low-ability  subjects 
degraded,  whereas  performance  of  the  high- 
ability  subjects  remained  approximately  the 
same.  Duncan  attributed  this  to  probable  dif¬ 
ferences  in  the  way  subjects  learned  the  task. 
He  hypothesized  that  those  people  with  good 
retention  had  learned  the  rules  governing  de¬ 
cision  tree  choices  (i.e.,  a  half-split  ap¬ 
proach),  whereas  those  with  poor  retention 
had  learned  the  series  of  steps  by  rote. 

Goldbeck  et  al.  (1957)  observed  differential 
effects  of  instructions  upon  performance,  de¬ 
pendent  upon  problem  characteristics  and 
individual  differences.  In  evaluating  the  ef¬ 
fects  of  different  types  of  instruction  upon 
subjects'  ability  to  solve  paper-and-pencil 
troubleshooting  problems,  they  noted  that 
instructions  on  how  to  perform  a  half-split 
test  were  sufficient  to  enable  subjects  to  solve 
simple  problems.  For  more  complex 
problems,  however,  differences  in  subjects’ 


performance  were  related  to  scores  on  a 
"problem-solving”  test  administered  at  the 
end  of  the  experiment.  Subjects  scoring  high 
on  that  test  were  still  able  to  solve  complex 
problems  with  the  half-split  instructions; 
low-scoring  subjects  were  able  to  solve  the 
complex  problems  only  if  given  additional 
instructions  and  practice  in  identifying  the 
feasible  set  of  alternatives. 

Elliott  (1965)  presented  high  school  boys 
with  performance  aids  varying  in  both  form 
(i.e.,  a  list  or  a  block  diagram)  and  level  of 
detail.  The  boys  then  attempted  to  use  the 
aids  to  solve  simulated  troubleshooting 
problems.  No  systematic  differences  in  speed 
were  observed,  but  boys  with  high  aptitude 
(scores  of  75-95  on  the  AFQT-F)  were  more 
accurate  than  those  with  medium  aptitude 
(scores  of  40-60).  There  was  also  an  interac¬ 
tion  of  aptitude  and  level  of  detail  of  the  aid; 
the  low-aptitude  group  had  particular  diffi¬ 
culty  in  using  the  aid  with  a  high  level  of  ex¬ 
traneous  detail. 

In  a  later  experiment,  Elliott  (1967)  pro¬ 
vided  high  school  juniors  with  a  trouble¬ 
shooting  guide  and  observed  them  as  they  at¬ 
tempted  to  solve  MTS-2  problems.  (The 
MTS-2  is  a  hardware  generic  electronic 
system.)  Subjects  were  classified  as  having 
high  or  medium  aptitude  on  the  basis  of  the 
AQE  (scores  of  80  —  95  and  50-65,  respec¬ 
tively).  High-aptitude  subjects  were  slightly 
better  at  correctly  isolating  faulty  modules 
and  components.  This  difference  was  attrib¬ 
uted  to  their  following  the  troubleshooting 
guide  more  closely. 

In  1968,  Elliott  and  Joyce  reviewed  the  re¬ 
sults  of  six  studies  investigating  variables  re¬ 
lated  to  the  design  and  use  of  job  perfor¬ 
mance  aids.  (The  two  previously  noted 
studies  by  Elliott  were  included  in  this  re¬ 
view.)  The  following  relationships  between 
aptitude  and  performance  were  noted.  First, 
speed  of  performance  did  not  appear  to  be  re¬ 
lated  to  aptitude  except  in  one  study  in- 
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volving  paper-and-pencil  troubleshooting 
tasks  and  "ambiguous”  test  results.  In  that 
study,  high-aptitude  subjects  (scoring  above 
75  on  the  EL  scale  of  the  AQE-F)  were  faster 
than  medium-aptitude  subjects  (scoring  be¬ 
tween  40  and  65).  Second,  high-aptitude  sub¬ 
jects  were  found  to  be  slightly  more  accurate 
than  medium-aptitude  subjects  in  solving 
paper-and-pencil  problems.  Finally,  no  dif¬ 
ferences  between  high-  and  medium-aptitude 
subjects  were  observed  in  their  ability  to 
troubleshoot  actual  equipment.  Elliott  and 
Joyce  attributed  the  failure  to  find  a  differ¬ 
ence  in  the  latter  case  to  the  variety  of  abili¬ 
ties  other  than  cognitive  skills  that  are  re¬ 
quired  in  actual  troubleshooting. 

An  alternative  explanation  for  the  general 
lack  of  a  clear  difference  between  high-  and 
medium-ability  troubleshooters  was  sug¬ 
gested  by  Henneman  and  Rouse  (1984).  In 
their  research,  ability  and  aptitude  were 
found  to  be  poor  predictors  of  performance  of 
a  simulated  troubleshooting  task,  although 
significant  regression  coefficients  were  ob¬ 
tained  when  entered  into  a  regression  equa¬ 
tion  with  cognitive  style  (discussed  later).  All 
of  the  subjects  in  this  research  were  students 
at  the  University  of  Illinois.  Their  interpreta¬ 
tion  was  that  troubleshooting  performance 
may  be  related  to  ability  only  if  some  min¬ 
imum  threshold  of  ability  is  not  present.  (Pre¬ 
sumably,  qualifying  for  admission  to  the  uni¬ 
versity  assured  that  the  threshold  was  ex¬ 
ceeded.)  Beyond  that  threshold,  other  factors 
are  more  important. 

Highland  et  aL  (1956)  administered  a  bat¬ 
tery  of  tests  to  360  experienced  radar  me¬ 
chanics  and  then  observed  their  performance 
as  they  attempted  to  solve  six  problems  on  an 
oscilloscope  kit.  Ratings  of  their  trouble¬ 
shooting  performance  were  then  compared 
with  test  results.  Highland  et  al.  found  that 
the  test  that  accounted  for  the  greatest  pro¬ 
portion  (60%)  of  the  variance  in  trouble¬ 
shooting  performance  assessed  "technical 


knowledge”  of  the  oscilloscope.  The  "elec¬ 
tronics  fundamentals”  test  accounted  for 
40%  of  the  variance,  and  a  test  of  "reasoning 
ability”  accounted  for  only  7%. 

A  study  by  Demaree,  Crowder,  and  Mor¬ 
rison  (1955)  is  presented  as  a  final  note  on 
aptitudes  and  abilities.  They  employed  a  va¬ 
riety  of  tests  to  assess  the  strengths  and 
weaknesses  of  Q-24  mechanics.  Of  the  many 
tests  they  used,  a  multiple-choice  test  of 
"practical  job  knowledge”  was  found  to  have 
the  highest  relationship  to  performance.  The 
aptitude  test  that  had  the  most  validity  was 
that  "requiring  the  mechanic  to  grasp  and 
find  the  consequences  of  complex  systems  of 
relationships.” 

Cognitive  styles.  A  number  of  measures  of 
cognitive  style  have  been  found  to  correlate 
with  troubleshooting  performance.  Two  di¬ 
mensions  of  cognitive  style  were  discussed 
most  often  in  the  studies  reviewed.  These 
were  field  dependence-independence  and  re- 
flectivity-impulsivity.  Rouse  and  his  col¬ 
leagues  (Henneman  and  Rouse,  1984;  Hunt, 
Henneman,  and  Rouse,  1981;  Rouse  and 
Rouse,  1979,  1982)  analyzed  the  results  of  re¬ 
search  in  which  subjects  solved  a  number  of 
troubleshooting  problems,  both  in  simple 
simulations  and  in  actual  equipment. 
Measures  of  cognitive  style  (scores  on  the 
Matching  Familiar  Figures  Test  and  the  Em¬ 
bedded  Figures  Test)  were  also  available  for 
these  subjects.  The  following  results  were  re¬ 
ported. 

First,  those  subjects  who  were  field  de¬ 
pendent  were  found  to  be  initially  slow  in 
.solving  simulated  troubleshooting  problems, 
but  became  faster  with  practice  until  there 
was  no  difference  between  field-dependent 
and  field-independent  subjects  with  regard 
to  the  amount  of  time  required  to  solve 
problems.  Second,  subjects  categorized  as 
impulsive  made  more  errors  in  trouble¬ 
shooting  than  did  those  who  were  reflective. 
The  impulsive  subjects  did  not  improve  with 
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practice.  This  finding  was  consistent  with 
other  reports  in  the  cognitive  style  literature. 
Third,  as  noted  earlier,  measures  of  ability 
(scores  on  the  ACT  test)  were  not  significantly 
correlated  with  measures  of  troubleshooting 
performance;  however,  significant  regression 
coefficients  were  found  for  measures  of 
ability  and  cognitive  style  when  the 
measures  were  combined  in  a  regression 
equation  to  predict  performance. 

Several  measures  of  cognitive  style,  abili¬ 
ties,  and  aptitudes  were  obtained  from  stu¬ 
dents  in  the  Navy’s  Basic  Electricity  and 
Electronics  (BE/E)  school  (Federico,  1982,' 
1983;  Federico  and  Landis,  1979,  1980).  The 
extent  to  which  these  measures  were  predic¬ 
tive  of  student  success  in  the  first  1 1  modules 
of  BE/E  school  was  assessed  via  correlations 
with  student  achievement  and  module  com¬ 
pletion  times.  For  7  of  the  11  modules, 
measures  of  cognitive  style  and/or  abilities 
were  found  to  be  more  predictive  than  were 
measures  of  aptitude.  Measures  of  ability 
were  more  closely  related  to  achievement  in 
the  early  modules,  and  other  measures  were 
more  predictive  of  performance  in  later  mod¬ 
ules.  With  respect  to  module  completion 
times,  measures  of  cognitive  style  and/or 
abilities  were  more  predictive  than  were 
measures  of  aptitude  in  all  modules.  Aptitude 
measures  were  more  highly  related  to  com¬ 
pletion  times  during  the  second  half  of  the 
course.  These  shifts  in  the  predictive  power  of 
each  measure  were  interpreted  as  reflecting 
changes  in  course  content. 

As  a  result  of  these  analyses,  it  was  con¬ 
cluded  that  module  completion  times  and 
achievement  in  BE/E  school  could  be  pre¬ 
dicted  from  combinations  of  the  measures 
employed.  For  example,  students  high  in 
achievement  were  found  to  be  discrimi¬ 
nating,  reflective,  high  in  general  and  logical 
reasoning  and  ideational  fluency,  and  good  at 
numerical  operations.  Rapid  completion 


times  were  found  to  be  associated  with  field 
independence,  high  general  and  inductive 
reasoning  ability,  and  high  ability  in  mathe¬ 
matics  and  general  science. 

Flexibility  of  closure  (i.e.,  ability  to  locate 
hidden  figures)  and  spatial  scanning  (i.e., 
speed  of  visual  exploration)  were  viewed  by 
Rose,  Fingerman,  Wheaton,  Eisner,  and 
Kramer  (1974)  as  abilities  rather  than  cogni¬ 
tive  styles,  and  together  they  were  found  to 
account  for  a  modest  but  significant  portion 
of  the  variance  in  performance  of  a  simple 
troubleshooting  task.  In  an  interesting  ap¬ 
proach,  an  attempt  was  made  to  improve 
these  abilities  in  subjects  through  training 
(Levine,  Brahlek,  Eisner,  and  Fleishman, 
1979;  Levine,  Schulman,  Brahlek,  and 
Fleishman,  1980).  Training  consisted  of  ex¬ 
tended  practice  with  paper-and-pencil  exer¬ 
cises  such  as  mazes  and  figure  copying. 
Training  resulted  in  a  difference  in  only  spa¬ 
tial  scanning  in  one  study,  and  no  subsequent 
differences  in  troubleshooting  were  noted. 

Summary.  Individual  differences  in  experi¬ 
ence,  cognitive  abilities  and  aptitudes,  and 
cognitive  style  have  been  shown  to  be  related 
to  troubleshooting  performance  in  a  variety 
of  ways.  The  following  statements  may  be 
made  about  the  effects  of  experience.  First,  in 
view  of  the  findings  reported  by  Vineberg,  ex¬ 
perience  may  be  helpful  and  even  necessary  if 
the  acquisition  of  essential  skills  requires  ex¬ 
tensive  practice,  or  if  the  opportunity  to 
practice  some  skills  occurs  infrequently. 
Second,  the  ability  to  learn  from  prototypical 
examples  appears  to  be  related  to  the  experi¬ 
ence  one  has  in  a  domain.  Third,  experience 
may  also  improve  the  ability  one  has  to  learn 
from  feedback  related  to  actions.  Finally, 
lack  of  experience  has  less  impact  upon  per¬ 
formance  as  more  guidance  in  trouble¬ 
shooting  is  provided. 

The  results  relevant  to  abilities  and  apti¬ 
tudes  may  be  summarized  as  follows.  First, 
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high  cognitive  ability  may  enable  one  to 
learn  more  from  prototypical  examples. 
Second,  high  cognitive  ability  is  associated 
with  a  greater  ability  to  employ  a  half-split 
approach  on  complex  problems.  Third, 
ability-related  differences  in  performance  are 
rather  small  if  comparisons  are  confined  to 
persons  having  at  least  some  moderate  level 
of  ability.  Finally,  the  measures  of  ability 
that  have  the  highest  relationship  to  perfor¬ 
mance  are  those  that  test  job-related  knowl¬ 
edge  and  skills  directly;  measures  of  general 
aptitude  are  not  very  predictive  of  problem¬ 
solving  performance  on  actual  equipment. 

Regarding  the  impact  of  cognitive  style  on 
troubleshooting,  several  measures  of  cogni¬ 
tive  style  have  been  reported  to  be  predictive 
of  success  in  electronics  training  courses. 
When  the  relationship  of  cognitive  style  to 
troubleshooting  performance  was  investi¬ 
gated,  reflective  subjects  were  found  to  incur 
less  unnecessary  cost.  Field-dependent  sub¬ 
jects  were  initially  slower  at  troubleshooting 
but  became  faster  with  practice.  Impulsive 
subjects  made  more  errors  and  did  not  im¬ 
prove  with  practice.  Finally,  training  to  im¬ 
prove  flexibility  of  closure  and  spatial  scan¬ 
ning  did  not  yield  the  anticipated  positive 
transfer  to  troubleshooting. 

APPROACHES  TO  TRAINING 

With  this  perspective  as  a  frame  of  refer¬ 
ence,  consider  the  various  training  ap¬ 
proaches  that  have  been  tried  and  the  effects 
these  approaches  have  had  upon  trouble¬ 
shooting  performance.  In  selecting  material 
for  inclusion  in  this  review,  only  those  re¬ 
ports  that  presented  data  relative  to  perfor¬ 
mance  effects  were  used.  This  criterion  was 
adopted  because  the  goal  of  this  paper  is  to 
summarize  the  state  of  empirical  knowledge 
about  troubleshooting  and  not  to  review  the 
many  approaches  to  troubleshooting 
training. 


One  way  in  which  the  studies  reviewed 
may  be  categorized  is  in  terms  of  the  primary 
rationale  or  goal  of  the  training  approach,  al¬ 
though  explicit  statements  of  this  were  rarely 
found.  Using  this  basis,  it  is  possible  to  iden¬ 
tify  four  general  approaches  to  training,  with 
a  salient  dimension  being  the  degree  to 
which  the  troubleshooter  is  explicitly  told 
what  to  do.  These  include:  (1)  instruction  in 
the  theory  upon  which  the  system  is  based, 
(2)  provision  of  opportunities  for  trouble¬ 
shooting  practice,  (3)  guidance  in  the  use  of 
system  knowledge,  and  (4)  guidance  in  the 
use  of  algorithms  or  rules. 

In  one  respect,  this  order  of  presentation  of 
training  approaches  may  seem  inappro¬ 
priate.  It  is  possible  to  view  theoretical  in¬ 
struction  and  opportunity  for  practice  as  op¬ 
posites  in  terms  of  system-relevant  informa¬ 
tion  supplied  to  the  troubleshooter.  The  focus 
of  theoretical  instruction  is  description  of  the 
functioning  of  the  system;  on  the  other  hand, 
it  is  possible  to  provide  an  opportunity  for 
troubleshooting  practice  without  ever  dis¬ 
cussing  the  nature  of  the  system.  These  two 
approaches  are  similar,  however,  with  regard 
to  the  action-related  information  provided.  In 
both  cases,  the  troubleshooter  is  expected  to 
develop  an  appropriate  strategy  rather  than 
to  follow  a  prescribed  strategy.  In  the  latter 
two  approaches  noted  (guidance  in  using 
knowledge  and  in  following  algorithms  or 
rules),  the  troubleshooter  is  given  more  infor¬ 
mation  about  how  to  proceed. 

Theoretical  Instruction 

In  this  very  traditional  approach,  the  trou¬ 
bleshooter  is  given  a  thorough  description  of 
the  functioning  of  the  system  and  instruction 
in  the  theory  upon  which  the  system  is  based. 
Once  provided  with  this  information,  the 
troubleshooter  is  expected  to  develop  an  ap¬ 
propriate  strategy  for  troubleshooting  the 
system.  In  many  of  the  studies  summarized 
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here,  the  performance  of  troubleshooters  in¬ 
structed  in  theory  was  compared  with  that  of 
a  control  group  of  persons  instructed  in  some 
other  way.  Usually  this  control  treatment 
consisted  of  some  form  of  proceduralization. 
Because  proceduralization  is  the  topic  of  dis¬ 
cussion  in  a  later  section,  it  is  not  discussed 
at  .length  here. 

Schorgmayer  and  Swanson  (1975)  varied 
the  nature  of  written  instructions  given  to 
trainees  in  electronics  troubleshooting.  Some 
trainees  were  given  a  "conventional”  account 
of  the  functioning  of  the  system,  and  others 
were  given  procedural  instructions,  with  no 
reference  to  the  way  in  which  the  system 
worked.  The  investigators  found  no  differ¬ 
ences  in  troubleshooting  abilities  of  the  two 
groups  in  familiar  or  unfamiliar  situations 
and  no  differences  in  attitude  toward  the  job. 

Shepherd,  Marshall,  Turner,  and  Duncan 
(1977)  observed  process  operator  trainees’ 
diagnostic  behavior  after  providing  them 
with  one  of  three  types  of  instruction:  (1)  no 
story,  consisting  merely  of  a  description  of 
the  interpretation  of  displays;  (2)  theory,  in¬ 
cluding  a  presentation  of  the  theory  upon 
which  the  system  was  based;  (3)  heuristics, 
including  rules-of-thumb  for  use  in  diagnosis. 
Of  the  three  groups,  the  heuristics  group  was 
better  in  both  familiar  and  unfamiliar  situa¬ 
tions.  The  theory  group  performed  no  better 
than  did  the  no-story  group. 

Miller  (1975)  described  research  in  which 
students  receiving  a  traditional  theoretical 
presentation  were  compared  with  students 
provided  with  a  function-  and  action-or¬ 
iented  presentation.  (A  more  complete  de¬ 
scription  of  this  study  may  be  found  in  the 
section  on  guidance  in  the  use  of  context-spe¬ 
cific  knowledge.)  Students  with  the  theory- 
based  course  were  slower,  made  more  errors, 
and  were  less  often  successful  in  trouble¬ 
shooting. 

Van  Matre  and  Steinemann  (1966)  and 


Steinemann,  Harrigan,  and  Van  Matre  (1967) 
presented  the  results  of  an  experimental 
training  program  in  electronics  for  Navy  per¬ 
sonnel  whose  aptitude  scores  were  too  low 
for  admission  into  the  regular  training  pro¬ 
gram.  The  course  of  instruction  in  this  pro¬ 
gram  was  altered  to  include  less  theory  and 
more  job-related  information.  On  most  de¬ 
pendent  measures  of  troubleshooting  perfor¬ 
mance,  the  experimental  group  was  equal  or 
superior  to  a  control  group  of  regular  stu¬ 
dents  completing  the  normal  training  course. 
These  measures  included  overall  perfor¬ 
mance,  instructor  and  peer  ratings  of  ability, 
and  time  required  to  troubleshoot.  However, 
the  experimental  group  scored  lower  on  a 
test  of  system  knowledge  and  indicated  a  de¬ 
sire  for  more  theoretical  instruction.  In  a 
subsequent  follow-up  study,  in  which  grad¬ 
uates  of  the  experimental  training  program 
were  compared  with  a  group  of  average 
trainees,  more  of  the  experimental  graduates 
were  successful  in  finding  faults. 

Williams  and  Whitmore  (1959)  conducted 
a  longitudinal  study  of  electronics  techni¬ 
cians.  Knowledge  of  theory  was  greatest  im¬ 
mediately  following  their  training  program 
and  was  lowest  when  the  follow-up  test  was 
given  three  years  later.  On  the  other  hand, 
troubleshooting  ability  was  worst  immedi¬ 
ately  following  training,  and  best  at  the  time 
of  the  final  follow-up  test. 

Foley  (1977)  presented  the  results  of  seven 
studies  (including  that  of  Williams  and  Whit¬ 
more),  in  which  job  performance  had  been 
compared  with  performance  on  theory  tests 
and  job  knowledge  tests.  Correlations  ob¬ 
tained  ranged  from  0.02  to  0.55,  with  a  mean 
of  0.2 1 .  Correlations  with  job  knowledge  tests 
were  slightly  higher  than  with  theory  tests. 

Summary.  The  results  of  these  studies  indi¬ 
cate  that  instruction  in  theoretical  principles 
is  not  an  effective  way  to  produce  good  trou¬ 
bleshooters.  It  is  interesting  to  note  that 
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these  results  are  quite  consistent  with  reports 
from  other  domains  such  as  process  control 
(Brigham  and  Laios,  1975;  Crossman  and 
Cooke,  1974;  Kragt  and  Landeweerd,  1974; 
Morris  and  Rouse,  1985)  and  mathemat¬ 
ical  problem  solving  (Mayer  and  Greeno, 
1972;  Mayer,  Stiehl,  and  Greeno,  1975),  in 
which  explicit  training  in  theories,  funda¬ 
mentals,  or  principles  failed  to  enhance  per¬ 
formance  and  sometimes  actually  degraded 
performance.  Fundamental  understanding 
has  been  empirically  shown  to  be  useful  for 
answering  theoretical  questions  but  not  for 
solving  problems. 

Opportunity  for  Practice 

The  essence  of  this  approach  is  to  provide 
the  troubleshooter  with  opportunities  to 
practice  solving  problems  using  the  actual 
system  or  some  form  of  simulation.  As  noted 
in  the  section  on  the  role  of  experience,  it  has 
often  been  said  that  practice  makes  perfect. 
This  approach  is  seldom  used  in  isolation  and 
is  usually  used  in  conjunction  with  some 
theoretical  instruction.  (In  fact,  all  of  the 
subjects  in  the  studies  summarized  in  this 
section  had  received  such  instruction  prior  to 
participating  in  the  research.)  We  have  in¬ 
cluded  it  as  a  separate  category  here  because 
in  some  of  the  studies  reviewed,  the  opportu¬ 
nity  for  practice  was  a  primary  goal  of  the 
development  of  the  device  or  method  investi¬ 
gated.  As  with  the  first  approach,  the  devel¬ 
opment  of  an  appropriate  strategy  may  be 
left  to  the  troubleshooter. 

The  opportunity  for  practice  seems  to  have 
been  the  primary  rationale  behind  the  cre¬ 
ation  of  the  Generalized  Maintenance 
Trainer  Simulator  (GMTS),  now  known  as 
the  Electronics  Equipment  Maintenance 
Trainer  (EEMT).  The  EEMT  is  a  computer- 
based  interactive  simulator  that  uses  video- 
disc-based  images  of  the  system  being  simu¬ 


lated.  The  images  depict  what  the  trouble¬ 
shooter  would  see  if  the  actual  system  were 
examined.  The  troubleshooter  solves 
problems  by  manipulating  switches  and  con¬ 
trols  via  a  touch-sensitive  device  and  then 
observing  the  effect  of  actions  upon  equip¬ 
ment  state.  "Tests"  are  also  performed  as 
necessary,  using  simulated  test  equipment 
readings,  until  single  faults  are  located. 

The  findings  of  the  GMTS/EEMT  studies 
may  be  summarized  as  follows.  In  the  early 
studies,  as  students  practiced  with  GMTS 
they  became  faster  and  made  fewer  replace¬ 
ments  in  subsequent  GMTS  problems.  These 
trends  may  reflect  a  learning  process  that 
was  taking  place;  however,  no  statistical 
tests  were  performed  in  these  studies 
(Rigney,  Towne,  King,  and  Moran,  1978; 
Rigney,  Towne,  Moran,  and  Mishler,  1980). 
In  a  later  study,  the  performance  of  students 
allowed  to  practice  with  either  the  actual 
equipment  or  GMTS  only  was  compared 
(Towne,  Munro,  Johnson,  and  Lahey,  1983). 
All  students  had  the  same  number  of  practice 
problems.  When  tested  on  actual  equipment, 
there  were  no  differences  between  the  two 
groups  in  number  of  replacements  or  solu¬ 
tion  times. 

Subsequent  studies  using  EEMT  (Cicchin- 
elli,  Keller,  and  Harmon,  1984)  also  reported 
improvements  in  troubleshooting  perfor¬ 
mance.  When  EEMT  was  made  available  to 
students  for  extra  practice,  those  students 
with  access  to  EEMT  demonstrated  an  11% 
gain  in  troubleshooting  performance  on  ac¬ 
tual  equipment.  When  students  in  another 
study  solved  the  same  number  of  problems 
either  with  EEMT  or  with  operational  equip¬ 
ment,  there  were  no  differences  on  a  perfor¬ 
mance  test.  Those  students  who  practiced  on 
actual  equipment  were,  however,  rated  more 
highly  by  laboratory  instructors.  In  a  follow¬ 
up  investigation  of  students  participating  in 
this  study,  there  were  no  differences  between 
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training  groups  with  regard  to  the  need  for 
later  training. 

Opportunity  for  practice  was  also  an  im¬ 
portant  feature  in  research  by  Johnson  and 
Rouse  (1982a,  1982b).  Three  groups  of  stu¬ 
dents  in  a  maintenance  training  course 
solved  simulated  problems  in  either  a  con¬ 
text-free  or  context-specific  environment,  or 
watched  videotaped  demonstrations  of 
troubleshooting  on  actual  equipment.  Subse¬ 
quent  performance  in  troubleshooting  actual 
equipment  was  roughly  equivalent  for  the 
three  groups.  (This  research  is  reviewed  in 
greater  detail  in  the  section  discussing  the  ef¬ 
fects  of  action-related  feedback.) 

A  form  of  guided  practice  was  described  in 
a  study  from  the  area  of  process  control. 
Duncan  and  Shepherd  (1975),  using  a  mock- 
up  of  a  control  room  operating  panel,  simu¬ 
lated  faults  in  a  process  plant  by  altering  dis¬ 
plays  of  state  variables.  Operator  trainees 
then  practiced  diagnosing  the  faults  via  a 
"cumulative  part"  training  technique,  iden¬ 
tifying  those  features  that  served  to  distin¬ 
guish  faults  from  each  other.  Duncan  and 
Shepherd  reported  a  general  trend  toward 
faster  diagnosis  with  practice,  but  no  statis¬ 
tical  tests  were  performed. 

Summary.  Considering  the  results  of  these 
studies,  it  may  be  stated  that  practice  in 
troubleshooting  improves  performance.  In 
some  cases  (e.g.,  GMTS  versus  actual  equip¬ 
ment),  the  opportunity  for  practice  was 
found  to  be  more  important  than  the  mode  of 
practice.  Of  course,  the  extent  to  which  this 
may  be  true  depends  on  the  characteristics  of 
the  simulation  used.  (See  the  later  section  on 
simulators  versus  actual  equipment  for  an 
elaboration  of  this  issue.) 

As  noted,  opportunity  for  practice  is 
seldom  used  in  isolation  as  a  training  tech¬ 
nique.  Having  the  trainee  "discover"  requi¬ 
site  knowledge  and  skills  would  seem  intu¬ 
itively  to  be  an  inefficient  and  time-con¬ 
suming  process,  particularly  for  large  or 
complex  systems.  Opportunity  for  practice  is 


viewed  as  an  essential  part  of  a  training  pro¬ 
gram.  For  the  full  potential  benefits  of  prac¬ 
tice  to  be  realized,  however,  complementary 
instruction  should  also  be  provided. 

Guidance  in  the  Use  of 
Context-Specific  Knowledge 

A  third  approach  focuses  on  helping  the 
troubleshooter  use  his  or  her  context-specific 
knowledge.  Judging  from  the  reports  classi¬ 
fied  in  this  category,  a  typical  assumption  is 
that  the  troubleshooter  possesses  some 
knowledge  about  the  system,  such  as  the  way 
the  system  functions.  (This  can  be  contrasted 
with  knowledge  of  troubleshooting  algor¬ 
ithms  or  procedures,  which  is  discussed  in  a 
later  section.)  The  goal  of  this  approach  is  to 
create  conditions  in  which  the  troubleshooter 
tries  to  apply  this  knowledge  to  problem 
solving,  with  or  without  explicit  instructions 
to  do  so.  For  example,  a  troubleshooter  might 
be  told  to  generate  hypotheses  as  to  the  cause 
of  symptoms  prior  to  beginning  work. 

Evidence  suggests  that  encouraging  trou¬ 
bleshooters  to  plan  before  acting  can  enhance 
their  performance.  In  an  experiment  con¬ 
ducted  by  Moore  et  al.  (1955),  trainees  were 
required  to  list  the  possible  causes  of  failures 
prior  to  solving  mock-up  generator  problems. 
They  also  identified  critical  tests  of  those  hy¬ 
potheses  and  proceeded  to  troubleshoot 
using  the  outline  generated  as  a  result  of  this 
planning  process.  Students  required  to  pro¬ 
ceed  in  this  manner  solved  more  problems 
than  did  a  control  group  that  was  allowed  to 
use  any  approach  they  wished. 

Helping  students  to  organize  the  informa¬ 
tion  provided  to  them  may  influence  their 
troubleshooting  performance.  Miller  (1975) 
described  an  experimental  training  course 
for  radar  mechanics,  in  which  an  effort  was 
made  to  relate  instruction  in  system  func¬ 
tioning  to  actions  performed  during  trouble¬ 
shooting.  For  example,  system  behavior  was 
presented  in  terms  of  causal  sequences  rather 
than  a  more  traditional  left-to-right  presen- 
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tation  of  schematics.  Whenever  possible,  at¬ 
tempts  were  made  to  relate  schematics  to  the 
actual  equipment,  often  with  the  use  of  pho¬ 
tographs  of  the  system.  Students  were  en¬ 
couraged  to  organize  the  information  they 
were  given  and  to  think  of  related  activities 
(such  as  actions  required  to  perform  tests)  as 
chunks  rather  than  as  isolated  actions. 

A  control  group  received  instruction  in  the 
theory  upon  which  the  system  was  based  and 
a  left-to-right  presentation  of  schematics, 
with  no  extraordinary  attempt  to  relate  the 
information  to  actual  equipment.  Both 
groups  had  limited  access  to  the  actual  radar 
for  troubleshooting  practice.  The  experi¬ 
mental  group  was  faster  in  performing 
checks  and  adjustments,  was  more  often  suc¬ 
cessful  in  troubleshooting,  made  fewer  errors 
in  general,  and  scored  better  on  quizzes. 

Providing  troubleshooters  with  guidance 
in  how  to  analyze  symptoms  has  also  been 
shown  to  be  helpful.  Fattu  and  Mech  (1953) 
manipulated  the  nature  of  information  given 
to  students  prior  to  solving  gear  train 
problems.  One  group  received  only  standard 
operating  procedures.  A  second  group  was 
given  an  introduction  to  the  gear  train,  in¬ 
cluding  nomenclature  and  basic  functioning, 
in  addition  to  the  standard  operating  proce¬ 
dures.  A  third  group  received  all  of  this  infor¬ 
mation  plus  instruction  in  symptom  analysis. 
As  a  result  of  their  training,  the  third  group 
solved  more  problems  than  did  the  other 
groups.  Interestingly,  the  speed  with  which 
the  third  group  solved  problems  did  not  im¬ 
prove  over  time,  whereas  the  other  two 
groups  became  faster. 

Perhaps  the  most  ambitious  attempt  to 
provide  guidance  in  the  use  of  system  knowl¬ 
edge  has  been  the  creation  of  SOPHIE 
(SOPHisticated  .instructional  Environment). 
SOPHIE  is  an  intelligent  interactive  com¬ 
puter-based  instructional  system  used  for 
teaching  electronics  troubleshooting.  The  au¬ 
thors  of  SOPHIE  have  been  quite  explicit 
in  stating  the  rationale  behind  their  work. 


SOPHIE  is  designed  with  the  goal  of  helping 
students  to  obtain  a  teleological  under¬ 
standing  of  their  system  and  represents  an 
attempt  to  provide  an  opportunity  for  stu¬ 
dents  to  apply  their  knowledge  and  “exercise 
their  logical  muscles.” 

Brown,  Rubenstein,  and  Burton  (1976)  de¬ 
scribed  two  possible  uses  of  SOPHIE.  One  is 
a  team  troubleshooting  game,  in  which  pairs 
of  players  seek  to  outwit  each  other  by  figura¬ 
tively  inserting  faults  into  a  simulated  elec¬ 
tronics  system.  The  other  involves  observa¬ 
tion  of  and  interaction  with  an  expert  trou¬ 
bleshooter  as  it  solves  student-initiated 
problems.  Analysis  of  verbal  protocols  taken 
from  people  interacting  with  SOPHIE  in  each 
of  these  scenarios  indicates  that  people  do  in¬ 
deed  appear  to  call  upon  their  system  knowl¬ 
edge  as  expected.  Response  from  users  of 
SOPHIE  has  also  been  favorable.  It  is  unfor¬ 
tunate  that  there  is  no  transfer  data  available 
to  indicate  that  forcing  people  to  use  system 
knowledge  in  this  way  does  actually  enable 
them  to  be  better  troubleshooters.  However, 
the  concept  is  intuitively  appealing. 

Summary.  When  one  compares  these  re¬ 
sults  with  those  reported  in  the  section  on 
theoretical  instruction,  it  becomes  clear  that, 
whereas  theoretical  instruction  alone  does 
not  appear  to  be  an  effective  approach  to 
troubleshooting  training,  teaching  people 
how  to  use  the  knowledge  they  have  can  lead 
to  better  performance.  It  should  be  noted 
that  the  studies  in  which  positive  effects  were 
found  involved  guidance  that  was  rather  ex¬ 
plicit:  students  were  told  to  generate  hypoth¬ 
eses,  chunk  information,  and  analyze 
symptoms  in  a  prescribed  way.  This  is  dif¬ 
ferent  from  providing  an  opportunity  to  use 
system  knowledge  and  should  not  be  inter¬ 
preted  as  evidence  that  such  an  approach 
will  produce  better  troubleshooters.  The  im¬ 
portant  question,  whether  exercising  one’s 
logical  muscles  enables  one  to  troubleshoot 
more  effectively,  remains  unanswered  and 
should  be  investigated. 
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Guidance  in  the  Use  of  Algorithms  or  Rules 

This  category  of  training  approaches  in¬ 
cludes  those  that  attempt  to  provide  the  trou¬ 
bleshooter  with  a  particular  strategy  or  at 
least  constrain  the  set  of  strategies  used.  This 
is  the  largest  category  of  the  four  discussed  in 
this  report,  and  it  encompasses  a  wide  range 
of  studies.  Typically,  studies  in  this  category 
attempted  to  constrain  the  troubleshooter's 
strategy  in  one  of  four  ways. 

First,  some  researchers  were  not  explicit  in 
telling  the  troubleshooter  what  should  or 
should  not  be  done,  but  provided  an  example 
of  the  “'right"  way  to  do  things  and  observed 
troubleshooting  behavior  to  see  if  this  ex¬ 
ample  was  followed.  Second,  troubleshooters 
were  sometimes  supplied  with  feedback  as  to 
the  acceptability  of  their  actions,  but  the  cri¬ 
teria  for  acceptable  actions  were  not  stated.  A 
third  approach  was  to  tell  the  troubleshooter 
about  appropriate  algorithms  or  rules  and 
observe  him  or  her  to  determine  whether  or 
not  the  approach  was  followed.  Finally,  in 
some  cases  the  task  of  troubleshooting  was 
fully  proceduralized.  Although  it  would  be 
more  appropriate  to  call  this  approach 
"aiding"  rather  than  "training,"  procedur- 
alization  is  discussed  here  for  completeness. 

Provision  of  examples .  One  illustration  of 
the  provision  of  models  has  been  noted  in  the 
discussion  of  SOPHIE  and  its  capability  to 
allow  students  to  observe  an  expert  trouble¬ 
shooter  in  action.  The  expert  troubleshooter 
solves  problems  according  to  an  optimal 
strategy.  In  that  discussion,  the  lack  of 
transfer  data  was  noted. 

Access  to  an  "expert"  is  also  a  salient  fea¬ 
ture  of  the  Adaptive  Computerized  Training 
System  (ACTS)  (Crooks,  Kuppin,  and  Freedy, 
1977;  Hopf-Weichel,  Purcell,  Freedy,  and  Lu- 
caccini,  1979).  Users  of  ACTS  may  trouble¬ 
shoot  a  computer-based  simulation  of  a 
power  supply  by  entering  commands  to  make 
tests,  replace  components,  etc.  Expert  recom¬ 
mendations  as  to  a  course  of  action  may  be 


obtained  from  a  model  based  upon  maxi¬ 
mizing  the  expected  value  or  utility  of  ac¬ 
tions.  Data  from  two  experiments  with  ACTS 
were  available. 

In  the  first  experiment,  six  college  students 
received  one  of  three  experimental  treat¬ 
ments.  One  group  selected  actions  from  a  list 
of  three  alternatives  supplied  by  the  expert 
model  and  received  feedback  as  to  which  of 
the  alternatives  would  have  been  the  expert's 
choice.  A  second  group  received  only  the  ex¬ 
pert's  feedback  after  determining  their  own 
actions.  Both  of  these  groups  had  the  oppor¬ 
tunity  to  change  their  actions  after  receiving 
feedback  from  the  expert.  A  third  group  did 
not  have  access  to  the  expert  at  all.  Expert 
aiding  was  available  in  two  sessions  and  was 
withdrawn  in  the  third  session.  The  first  two 
groups  achieved  lower-cost  solutions  to 
problems  when  the  aiding  was  present,  but 
there  were  no  differences  in  the  three  groups 
during  the  third  session  when  no  aiding  was 
available.  This  conclusion  is  based  upon 
means  only,  as  there  were  no  statistical  tests 
performed.  Additionally,  the  test  of  transfer 
to  an  unaided  situation  was  confounded  by 
the  fact  that  the  problems  solved  during  the 
third  (unaided)  session  were  identical  to 
those  solved  in  the  first  session. 

The  second  experiment  with  ACTS  in¬ 
volved  three  college  students  solving 
problems  in  two  sessions.  It  was  reported 
that  the  expert  model  converged  with  the 
performance  of  two  subjects  by  the  end  of  the 
second  session.  As  with  the  first  experiment, 
no  statistical  tests  were  performed,  and  there 
was  no  test  of  transfer  to  unaided  situations, 
so  the  effectiveness  of  this  technique  as  a 
training  approach  cannot  be  assessed. 

There  is  limited  evidence  available,  how¬ 
ever,  that  provision  of  a  model  approach  can 
lead  to  improved  troubleshooting  perfor¬ 
mance.  Duncan  (1971)  created  a  mock  trou¬ 
bleshooting  task  consisting  of  a  chain  of 
block  components  with  various  probabilities 
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of  failure  and  provided  college  students  with 
a  decision  tree  aid  based  upon  an  optimal 
half-split  approach.  The  students  were  in¬ 
structed  to  troubleshoot  according  to  the  de¬ 
cision  tree  and  to  try  to  determine  the  ratio¬ 
nale  underlying  the  recommended  decisions. 
The  use  of  the  aid  led  to  improved  perfor¬ 
mance  for  all  subjects.  However,  as  noted  in 
an  earlier  section,  the  degree  to  which  this 
improvement  was  retained  in  the  absence  of 
the  aid  was  highly  related  to  ability,  as  mea¬ 
sured  by  pre-aiding  performance. 

Feedback  about  actions.  In  most  of  the 
studies  reviewed,  action  feedback  was  in  the 
form  of  bookkeeping,  that  is,  identifying  the 
feasible  set  of  remaining  alternatives  given 
the  results  of  tests  already  performed.  As  an 
illustration,  consider  research  in  which  col¬ 
lege  students  were  given  a  set  of  symptoms 
and  asked  to  identify  the  component 
common  to  those  symptoms  (Brooke, 
Duncan,  and  Cooper,  1980;  Brooke,  Duncan, 
and  Marshall,  1978).  For  example,  subjects 
were  asked  to  discover  which  "factory”  was 
discharging  pollutants  into  the  river,  given 
symptoms  such  as  "dead  fish”  and  "bad 
smell  ”  Problems  were  displayed  on  a  CRT, 
and  subjects  were  allowed  to  perform  actions 
such  as  checking  the  color  of  the  water  at 
certain  locations,  etc.,  by  entering  commands 
at  the  terminal.  After  each  action,  feedback 
displayed  at  the  screen’s  bottom  indicated 
the  set  of  alternatives  remaining,  given  the 
results  of  that  test. 

Two  experiments  were  performed  using 
this  approach.  In  the  first  experiment,  sub¬ 
jects  were  presented  with  three  types  of 
problems:  identification  of  (1)  famous  people, 
(2)  random  names,  or  (3)  factories.  One  group 
received  the  feedback  while  solving  the 
famous  people  and  random  names  problems 
and  then  solved  the  factories  problems 
without  feedback.  A  second  group  solved  the 
famous  people  and  factories  problems  with 
feedback  and  transferred  to  random  names 


without  feedback.  Groups  3  and  4  served  as 
control  groups  and  received  the  same  treat¬ 
ment  as  Groups  1  and  2,  respectively,  with 
the  exception  that  no  information  about  the 
feasible  set  of  alternatives  was  ever  provided. 
Surprisingly,  there  was  no  positive  transfer 
for  the  first  group  when  feedback  was  re¬ 
moved,  but  there  was  for  the  second. 

The  experimental  procedure  was  essen¬ 
tially  the  same  for  the  second  experiment, 
with  the  exception  that  problems  involving 
cities  were  substituted  for  the  famous  people 
problems.  As  a  result,  identification  of  the 
feasible  set  led  to  positive  transfer  for  both 
experimental  groups.  These  findings  led 
Brooke  et  al.  to  conclude  that  feedback  about 
the  set  of  possible  faults  may  produce  posi¬ 
tive  transfer,  but  only  if  the  feedback  is  pro¬ 
vided  in  a  variety  of  situations  during 
training. 

Aiding  in  the  form  of  action-related  feed¬ 
back  was  also  investigated  in  a  series  of  ex¬ 
periments  by  Rouse  and  his  colleagues.  A  full 
discussion  of  each  experiment  conducted 
would  be  rather  lengthy  and  unnecessary  for 
this  paper,  so  the  reader  is  referred  to  Rouse 
and  Hunt  (1984)  for  a  detailed  description  of 
the  10  experiments  reviewed.  For  present 
purposes,  a  summary  of  the  general  approach 
and  major  results  will  suffice. 

Troubleshooting  performance  was  investi¬ 
gated  in  the  context  of  two  experimental 
computer-based  tasks.  The  first,  Trouble¬ 
shooting  by  Application  of  Structural  Knowl¬ 
edge  (TASK),  is  a  context-free  environment  in 
which  the  troubleshooter  is  shown  a  pattern 
of  system  outputs  in  the  form  of  zeros  and 
ones  and  is  given  the  problem  of  locating  the 
faulty  component  by  testing  connections  be¬ 
tween  components.  Two  versions  of  TASK 
have  been  used.  TASK1  contains  no  feedback 
loops,  and  consists  solely  of  AND  gates. 
TASK2  contains  both  AND  and  OR  gates,  and 
includes  feedback  loops. 

The  second  experimental  task  used  is 
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FAULT  (Framework  for  Aiding  the  Under¬ 
standing  of  Logical  Troubleshooting).  FAULT 
is  similar  to  TASK  in  that  it  consists  of  a  net¬ 
work  of  components  but,  unlike  TASK,  con¬ 
tains  a  great  deal  of  context.  A  number  of 
systems  have  been  simulated  in  the  FAULT 
framework,  including  automobile  engines 
and  a  variety  of  aircraft  engines.  The  trouble¬ 
shooter  attempts  to  solve  FAULT  problems 
by  entering  commands  to  make  tests,  replace 
parts,  etc. 

Feedback  or  aiding  in  most  of  the  TASK 
and  FAULT  research  has  consisted  of  identi¬ 
fication  of  the  set  of  possible  faulty  compo¬ 
nents  in  TASK1  problems.  Several  positive 
transfer  effects  have  been  noted  as  a  result  of 
aided  practice  on  TASK1.  For  example,  sub¬ 
jects  provided  with  aiding  during  training  on 
TASK1  made  fewer  tests  to  solution  on  sub¬ 
sequent  unaided  TASK1  problems;  there 
was,  however,  an  increase  in  time  required 
during  the  first  unaided  problems.  Interest¬ 
ingly,  after  practicing  on  aided  TASK1,  time 
to  solve  TASK2  problems  was  initially  longer 
than  for  a  control  group  that  received  no 
aiding  during  practice.  However,  time  to 
solve  TASK2  problems  eventually  decreased, 
and  aided  subjects  became  faster  than  those 
in  the  control  group. 

Aiding  on  TASK1  led  to  differences  in  the 
amount  of  information  obtained  per  action 
when  solving  FAULT  problems.  These  effects 
were  also  dependent  on  experience.  Positive 
transfer  effects  were  obtained  for  students  in 
the  fourth  semester  of  an  aircraft  mainte¬ 
nance  training  course,  whereas  negative  ef¬ 
fects  were  obtained  for  first-semester 
trainees.  Finally,  aiding  on  TASK1  had  a 
beneficial  effect  on  the  number  of  inefficient 
actions  (compared  with  an  optimal  solution) 
made  in  solving  FAULT  problems.  This  effect 
was  greater  when  transferring  to  problems  in 
unfamiliar  FAULT  systems. 

The  effects  of  practice  with  aided  TASK1 


and  FAULT  upon  troubleshooting  of  actual 
equipment  were  investigated  in  two  experi¬ 
ments  conducted  by  Johnson  and  Rouse 
(1982a,  1982b).  Subjects  were  students 
enrolled  in  the  last  semester  of  an  aircraft 
maintenance  training  course.  Three  ap¬ 
proaches  to  training  were  compared  in  the 
first  study.  One  group  of  subjects  solved  60 
aided  TASK1  problems.  A  second  group 
solved  35  FAULT  problems,  including 
problems  with  an  automobile  engine  and  two 
aircraft  engines.  A  third  group  was  given  a 
reading  assignment  on  troubleshooting  and 
watched  videotapes  demonstrating  the 
troubleshooting  of  three  faults  on  actual 
equipment.  Training  effectiveness  was  evalu¬ 
ated  by  observing  trainees  as  they  solved  five 
problems  on  actual  equipment. 

The  results  of  this  experiment  were  as 
follows.  First,  there  were  no  differences  be¬ 
tween  groups  with  regard  to  time  to  solve 
problems.  Second,  the  performance  of  those 
students  who  watched  the  videotaped  trou¬ 
bleshooting  demonstrations  was  rated  higher 
overall  by  instructors.  Third,  ratings  of  the 
quality  of  the  sequences  of  tests  made  were 
comparable  for  all  groups,  with  one  excep¬ 
tion.  Three  of  the  problems  solved  during  the 
evaluation  had  been  demonstrated  in  the  vi¬ 
deotaped  lessons;  students  who  had  seen  the 
videotapes  achieved  higher  quality  ratings 
on  those  problems. 

Analysis  of  errors  made  by  students  while 
troubleshooting  indicated  that  those  stu¬ 
dents  who  practiced  with  FAULT  made  sig¬ 
nificantly  more  errors  than  did  the  videotape 
group  in  the  use  of  test  equipment,  and  that 
those  who  practiced  with  TASK1  made  little 
use  of  test  equipment  at  all.  In  light  of  these 
results,  FAULT  was  modified  to  include  an 
explanation  of  the  use  of  test  equipment,  and 
the  second  study  was  conducted.  In  this  ex¬ 
periment,  a  videotape  group  was  compared 
to  a  group  receiving  practice  on  both  aided 
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TASK1  and  the  modified  FAULT.  No  signifi¬ 
cant  differences  between  the  two  groups  were 
observed. 

Experiments  conducted  by  Hunt  (1981) 
and  Johnson  and  Fath  (1984)  investigated  the 
effects  of  another  form  of  action-related  feed¬ 
back.  In  these  studies,  aiding  was  introduced 
into  the  FAULT  environment.  Whenever  a 
test  was  made  that  was  inappropriate  in 
light  of  known  symptoms  or  the  results  of 
previous  tests,  subjects  received  messages  ex¬ 
plaining  the  reasons  why  they  should  not 
have  needed  to  make  that  test.  These  mes¬ 
sages  were  explicit  and  highly  context-spe¬ 
cific.  Hunt  found  that  maintenance  trainees 
provided  with  this  error  feedback  made 
fewer  inferential  errors  on  subsequent 
FAULT  problems  and  that  this  effect  per¬ 
sisted  regardless  of  the  system  simulated.  In 
the  experiment  conducted  by  Johnson  and 
Fath,  performance  of  students  trained  using  a 
FAULT-based  simulation  was  compared  with 
the  performance  of  students  trained  with  ac¬ 
tual  equipment.  In  all  cases  the  performance 
of  the  FAULT-trained  students  was  found  to 
be  equal  or  superior  to  the  group  using  real 
equipment. 

Whereas  the  approaches  involving  feed¬ 
back  about  inferential  errors  and  the  set  of 
remaining  alternatives  have  been  shown  to 
have  positive  effects  upon  troubleshooting 
performance,  supplying  subjects  with  unex¬ 
plained  ratings  about  the  quality  of  their  ac¬ 
tions  has  had  undesirable  results.  Rouse, 
Rouse,  and  Pellegrino  (1980)  rated  subjects' 
actions  while  solving  TASK  problems  as 
being  excellent,  good,  fair,  or  poor.  Ratings 
were  based  upon  the  judged  quality  of  heur¬ 
istics  that  Rouse  et  al.  felt  could  have  been 
used  to  generate  the  actions.  Subjects  were 
given  no  information  about  the  source  of  the 
ratings.  Rating  actions  in  this  manner  led  to 
the  solution  of  fewer  problems  and  more  tests 
made  per  problem. 


Instruction  in  algorithms  or  rules.  With  the 
exception  of  one  study  reviewed,  the  algor¬ 
ithm  of  choice  was  a  half-split  approach, 
with  the  goal  of  minimizing  the  number  of 
tests  required  to  localize  the  source  of  a 
failure.  This  exception  was  a  study  by  Glass 
(1967),  in  which  students  heard  a  series  of 
taped  lectures  on  “systematic  trouble¬ 
shooting/'  instructing  them  to  use  a  tracing- 
back  approach.  As  a  result,  students  re¬ 
ceiving  lectures  were  more  successful  in 
troubleshooting  actual  equipment  than  were 
students  who  did  not  hear  the  lectures. 

Goldbeck  et  al.  (1957)  conducted  two 
studies  in  which  subjects  were  instructed  in 
the  use  of  a  half-split  approach.  The  trouble¬ 
shooting  task  employed  was  a  simple  simula¬ 
tion  consisting  of  a  network  of  block  compo¬ 
nents  and  outputs  of  zeros  and  ones.  In  the 
first  study,  the  independent  variables  were 
complexity  of  the  network  and  the  avail¬ 
ability  of  instructions  in  the  half-split  ap¬ 
proach.  Goldbeck  et  al.  reported  the  fol¬ 
lowing  effects  of  the  half-split  instructions. 
First,  persons  instructed  in  the  use  of  a  half¬ 
split  approach  made  fewer  tests  in  simple 
networks  prior  to  locating  the  failure,  but  not 
when  the  networks  were  more  complex.  They 
also  required  more  time  to  solve  problems, 
and  this  difference  increased  with  problem 
complexity.  Finally,  they  appeared  to  be  no 
better  than  the  control  group  at  identifying 
the  set  of  feasible  alternatives. 

Differential  effects  of  instructions  were 
also  investigated  in  the  second  study.  In  ad¬ 
dition  to  a  control  group  that  received  no  in¬ 
formation  about  the  half-split  approach, 
three  groups  received  variations  of  half-split 
instructions.  The  “half-split"  group  received 
practice  in  identifying  half-split  tests,  but  no 
practice  in  identifying  the  set  of  feasible  al¬ 
ternatives.  The  “deductive  half-split"  group 
received  instruction  in  the  concept  of  a  half¬ 
split,  plus  practice  with  complex  problems 
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and  the  opportunity  to  correct  errors  after 
feedback.  The  "deductive  alternatives"  group 
received  half-split  instructions  plus  a  discus¬ 
sion  of  alternative  (suboptimal)  ways  to 
eliminate  components  from  further  consider¬ 
ation. 

Although  there  were  no  differences  be¬ 
tween  groups  in  the  number  of  replacements 
made,  the  deductive  half-split  group  required 
fewer  tests  than  did  the  other  groups,  and 
their  first  checks  were  more  often  in  the  fea¬ 
sible  set  of  alternatives.  In  general,  the  per¬ 
formance  of  the  deductive  half-split  group 
was  superior  to  the  performance  of  the  exper¬ 
imental  group  in  the  first  study.  As  an  inter¬ 
esting  note,  the  first  checks  of  the  half-split 
group  were  in  the  feasible  set  less  often  than 
were  those  of  any  other  group. 

Goldbeck  et  al.  interpreted  their  results  as 
follows.  If  the  system  is  simple,  then  teaching 
troubleshooters  about  the  half-split  concept 
may  enhance  their  performance.  However,  if 
the  system  is  complex,  additional  instruction 
and  practice  in  identifying  the  feasible  set  of 
alternatives  may  be  required.  As  may  be  re¬ 
called  from  the  discussion  of  this  study  in  the 
section  on  individual  differences,  this  need 
for  elaboration  and  practice  may  be  less  for 
persons  with  high  ability. 

Brooke,  Cooke,  and  Duncan  (1983)  also  in¬ 
vestigated  the  value  of  instruction  in  the 
half-split  approach,  using  an  experimental 
task  similar  to  TASK  developed  by  Rouse. 
There  were  two  independent  variables  in 
their  research:  the  availability  of  strategic 
instructions,  and  the  availability  of  aiding. 
The  instructions  consisted  of  a  discussion  of 
the  concepts  of  half-split  and  bracketing  and 
a  caution  to  make  diagnoses  only  with  suffi¬ 
cient  information.  Aiding  involved  providing 
feedback  when  tests  made  were  redundant  or 
diagnoses  were  premature. 

Brooke  et  al.  reported  that  subjects  tended 
to  make  premature  diagnoses  if  not  provided 
instructions  or  aiding,  and  did  not  improve 
over  time.  Aiding  alone  was  somewhat 


helpful  in  alleviating  this  problem,  and  in¬ 
structions  were  initially  useful.  However,  the 
beneficial  effects  of  instructions  on  prema¬ 
ture  diagnosis  deteriorated  if  aiding  was  not 
also  available.  This  was  not  the  case  with  test 
efficiency,  where  a  general  overall  improve¬ 
ment  due  to  instructions  was  noted.  From 
these  results,  it  was  concluded  that  the  in¬ 
structions  were  generally  more  effective  in 
influencing  troubleshooting  strategies  than 
was  the  aiding,  and  that  the  efficiency  of 
testing  was  more  easily  improved  than  were 
diagnostic  errors. 

Explicit  instruction  in  heuristics  rather 
than  an  algorithm  was  the  approach  em¬ 
ployed  by  Shepherd  et  al.  (1977),  Marshall, 
Scanlon,  Shepherd,  and  Duncan  (1981),  and 
Marshall,  Duncan,  and  Baker  (1981).  The  re¬ 
search  by  Shepherd  et  al.  is  described  earlier, 
in  which  supplying  process  operator  trainees 
with  diagnostic  rules  was  found  to  be  more 
effective  than  providing  them  with  theoret¬ 
ical  instruction.  In  the  experiments  reported 
by  Marshall  and  colleagues,  greater  effort 
was  made  to  ensure  that  the  trainees  used  the 
rules  as  instructed  by  controlling  their  access 
to  system  information.  Trainees  received  in¬ 
formation  in  one  of  two  ways.  Either  infor¬ 
mation  was  displayed  all  at  once,  as  on  a 
conventional  control  room  operating  panel, 
or  trainees  had  to  request  information  as  de¬ 
sired  via  a  terminal  keyboard. 

By  thus  limiting  the  sequence  in  which  in¬ 
formation  was  accessed,  it  was  possible  to 
force  the  use  of  the  diagnostic  rules.  The  ef¬ 
fectiveness  of  the  method  was  evaluated  by 
observing  persons  trained  with  each  mode  of 
information  presentation  as  they  diagnosed 
faults  from  a  conventional  unrestricted  dis¬ 
play.  Those  persons  forced  to  use  the  diag¬ 
nostic  rules  during  training  were  more  suc¬ 
cessful  in  making  diagnoses. 

Proceduralization.  In  the  research  just  de¬ 
scribed,  troubleshooters  learned  a  list  of 
rules,  which  they  sought  to  apply  when 
making  diagnoses.  Rules  are  also  provided 
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when  the  task  of  troubleshooting  is  procedur- 
alized,  but  the  troubleshooter  is  not  expected 
to  memorize  them.  Rather,  the  procedures 
are  always  accessible,  either  on-line  or  in 
hard-copy  form,  and  are  retrieved  as  needed. 

There  is  little  doubt  that  supplying  people 
with  adequate  procedures  can  have  a  positive 
effect  on  their  troubleshooting  performance. 
This  has  been  demonstrated  in  several 
studies.  For  example,  Potter  and  Thomas 
(1976)  observed  technicians  as  they  per¬ 
formed  electronics  troubleshooting  tasks  and 
reported  differences  in  number  of  correct  so¬ 
lutions  and  time  to  solution  as  a  function  of 
the  level  of  specificity  of  the  job  performance 
aid  used.  Smillie  and  Porta  (1981)  and  Elliott 
(1966)  noted  improvements  in  the  number  of 
correct  solutions,  number  of  checks  and  un¬ 
necessary  replacements,  and  solution  time 
when  proceduralized  aids  were  used  rather 
than  more  traditional  approaches  such  as 
technical  orders. 

Most  of  the  reports  of  the  effects  of  proce¬ 
dures  were  written  by  Elliott  (1965,  1966, 
1967)  and  Elliott  and  Joyce  (1968,  1971)  and 
are  discussed  in  conjunction  with  the  effects 
of  ability  and  experience.  As  a  group,  these 
studies  may  be  viewed  as  indicating  that  pro- 
ceduralization  may  be  a  very  effective  means 
of  influencing  troubleshooting  performance, 
largely  independent  of  ability  or  experience. 
To  realize  this,  it  is  only  necessary  to  recall 
the  experiment  in  which  proceduralization 
enabled  high  school  students  with  no  trou¬ 
bleshooting  experience  to  find  faults  as  accu¬ 
rately  as  Air  Force  technicians. 

Summary.  In  light  of  the  results  presented 
in  this  section  on  guidance  in  the  use  of  al¬ 
gorithms  or  rules,  the  following  conclusions 
are  offered.  First,  the  limited  evidence  avail¬ 
able  suggests  that  providing  troubleshooters 
with  good  examples  can  have  a  beneficial  ef¬ 
fect  on  their  performance.  However,  learning 
from  examples  may  be  confined  to  persons 
with  high  ability.  It  may  also  be  noted  that 
subjects  in  the  one  study  upon  which  this 


conclusion  is  based  were  explicitly  instructed 
to  learn  from  the  examples  provided.  It 
seems  likely  that  in  the  absence  of  this  in¬ 
struction  less  learning  would  occur. 

Troubleshooting  performance  may  be  en¬ 
hanced  if  action-related  feedback  is  provided. 
If  feedback  is  in  the  form  of  identifying  the 
feasible  set  of  alternatives,  positive  transfer 
effects  may  be  noted  only  if  this  feedback  is 
supplied  in  a  variety  of  contexts  during 
training  or  if  the  troubleshooter  has  some 
level  of  prior  experience.  Explicit  feedback 
about  inferential  errors  has  also  been  shown 
to  improve  performance,  but  unexplained 
ratings  about  the  quality  of  actions  have  pro¬ 
duced  negative  results. 

Instruction  in  algorithms  has  been  found  to 
be  effective  if  (1)  the  algorithm  is  simple  (e.g., 
tracing  back),  or  (2)  the  system  is  simple.  If 
the  use  of  a  half-split  approach  is  desired  and 
the  system  is  not  simple,  then  it  may  be  nec¬ 
essary  to  provide  extended  instruction  and 
opportunity  for  practice  or  to  select  trouble¬ 
shooters  with  high  ability.  In  the  one  study  in 
which  they  were  compared,  instruction  in  the 
use  of  an  algorithm  was  found  to  be  more  ef¬ 
fective  in  improving  test  efficiency  than  was 
the  provision  of  feedback. 

Compared  with  theoretical  instruction,  the 
provision  of  heuristics  can  lead  to  improved 
troubleshooting  performance.  Greater  im¬ 
provement  is  realized  if  the  use  of  the  heur¬ 
istics  during  training  can  be  forced. 

Finally,  positive  effects  of  procedures  are 
possible,  even  with  troubleshooters  deficient 
in  ability  or  experience.  These  effects  depend 
to  some  extent  upon  the  form  and  specificity 
of  the  procedures.  Within  reason,  as  proce¬ 
dures  become  more  explicit,  less  ability  and 
experience  are  needed  for  comparable  perfor¬ 
mance. 

Simulators  versus  Actual  Equipment 

A  number  of  the  studies  reviewed  in  this 
report  employed  some  form  of  simulation  in 
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their  research.  Care  must  be  exercised  in  gen¬ 
eralizing  the  results  of  such  research  to  real 
equipment  performance  because  of  differ¬ 
ences  in  the  nature  of  the  simulations.  There 
have  been  several  instances  reported  in 
which  performance  using  a  simulator  dif¬ 
fered  from  performance  using  actual  equip¬ 
ment.  Because  simulators  are  often  employed 
in  training,  it  is  important  to  be  aware  of 
such  differences.  Due  to  their  implications 
for  training  programs,  studies  are  discussed 
here  that  compared  performance  on  simu¬ 
lators  with  performance  on  actual  equip¬ 
ment. 

Standlee  and  Bilinski  (1966)  tested  the 
troubleshooting  performance  of  114  data 
systems  technicians  with  both  a  paper-and- 
pencil  test  and  actual  equipment.  Although 
no  statistical  tests  were  presented,  Standlee 
and  Bilinski  reported  that  ratings  of  perfor¬ 
mance  on  the  two  tests  could  lead  one  to 
reach  opposite  conclusions.  Ratings  of  per¬ 
formance  with  the  paper-and-pencil  simula¬ 
tion  were  generally  low,  whereas  perfor¬ 
mance  using  actual  equipment  was  rated 
higher. 

Similar  results  were  reported  by  Stein- 
eman  (1966).  Student  technicians  solved 
identical  faults  using  either  a  paper-and- 
pencil  diagram  or  a  hardware  simulation  of 
actual  equipment.  Most  of  the  correlations 
between  performance  on  the  two  simulations 
were  negative.  More  replacements  were 
made  with  the  paper-and-pencil  simulation, 
and  use  of  the  hardware  simulation  was  asso¬ 
ciated  with  more  wrong  readings  and  re¬ 
checks. 

Low  correlations  between  measures  of  per¬ 
formance  on  simulated  or  actual  equipment 
were  noted  by  Shriver  and  Foley  (1974)  and 
Mallory  and  Elliott  (1978).  Both  of  these 
studies  compared  performance  on  a  photo- 
based  simulation  with  that  on  electronics 
equipment.  Mallory  and  Elliott  noted  corre¬ 
lations  between  performance  measures  such 


as  speed,  accuracy,  and  number  of  steps, 
ranging  from  0.35  to  0.58.  Shriver  and  Foley 
reported  higher  correlations  (ranging  from 
0.53  to  0.87)  dependent  upon  the  "level  of 
penetration  of  the  troubleshooting  task.  As 
performance  of  the  actual  task  required 
greater  use  of  tools  and  test  equipment 
(which  was  not  possible  with  their  simula¬ 
tion),  then  there  was  less  of  a  relationship  be¬ 
tween  performance  measures  on  simulated 
and  actual  equipment. 

Elliott  and  Joyce  (1968)  noted  a  relation¬ 
ship  between  aptitude  and  performance  of  a 
paper-and-pencil  troubleshooting  task  but 
failed  to  find  a  similar  relationship  when  the 
task  involved  was  troubleshooting  of  actual 
equipment.  Students  with  high  aptitude 
were  found  to  be  more  accurate  than  were 
those  of  medium  aptitude  in  troubleshooting 
a  paper-and-pencil  simulation,  but  no  differ¬ 
ences  were  found  in  their  troubleshooting  of 
actual  equipment.  The  explanation  provided 
was  similar  to  that  offered  by  Shriver  and 
Foley,  citing  differences  in  skills  required  to 
perform  the  two  tasks. 

Although  differences  in  performance  using 
simulators  or  actual  equipment  have  been 
noted,  those  studies  in  which  transfer  from 
simulators  to  actual  equipment  was  evalu¬ 
ated  reported  few  or  no  differences  in 
transfer  (Johnson  and  Fath,  1984;  Johnson 
and  Rouse,  1982b;  McDonald  et  al.,  1983; 
Towne  et  ah,  1983;  Cicchinelli  et  ah,  1984).  In 
each  of  these  studies,  the  opportunity  to 
practice  on  the  simulation  was  only  part  of  a 
training  program;  often,  practice  on  actual 
equipment  was  also  provided.  Therefore,  it  is 
difficult  to  assess  differences  that  might  have 
been  present  if  only  practice  on  the  simula¬ 
tions  had  been  allowed.  What  may  be  said  is 
that  in  each  of  these  training  programs  stu¬ 
dents  allowed  to  practice  on  the  simulation 
and  students  practicing  only  on  real  equip¬ 
ment  appeared  to  acquire  equivalent  trou¬ 
bleshooting  skills. 
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Summary.  Three  statements  may  be  made 
in  view  of  the  research  in  which  differences 
in  performance  with  simulators  and  actual 
equipment  were  found.  First,  the  relationship 
between  troubleshooting  performance  and 
characteristics  such  as  general  abilities  or 
aptitude  is  weaker  as  the  range  of  required 
abilities  increases.  Stronger  relationships 
may  be  found  with  simple  simulations  be¬ 
cause  performance  may  depend  on  a  few 
skills.  Second,  performance  on  a  simulation 
will  be  related  to  troubleshooting  of  actual 
equipment  to  the  extent  that  the  same  skills 
are  required.  Some  simulations  only  allow 
the  troubleshooter  to  practice  a  strategy.  If 
the  student  is  not  allowed  to  practice  making 
tests  and  repairing  or  replacing  components, 
then  performance  on  the  simulator  may  not 
be  highly  related  to  performance  on  actual 
equipment.  Finally,  these  results  underscore 
the  need  to  evaluate  transfer  of  training  in 
the  context  of  actual  equipment. 

CONCLUSIONS 

In  the  introduction,  we  stated  that  three 
abilities  are  necessary  for  success  in  trouble¬ 
shooting.  These  include:  (1)  the  ability  to  re¬ 
pair  or  replace  components,  (2)  the  ability  to 
perform  tests,  and  (3)  the  ability  to  employ  a 
strategy  in  attacking  the  problem.  Because 
the  third  of  these  abilities  poses  such  a  con¬ 
ceptual  challenge  to  instructors,  and  due  to 
the  nature  of  available  literature,  the  focus 
here  has  been  on  the  use  of  strategies.  How¬ 
ever,  it  is  necessary  to  reemphasize  the  im¬ 
portance  of  the  first  two  abilities  as  well. 
Teaching  persons  to  repair/replace  compo¬ 
nents  and  to  make  tests  should  be  included  in 
a  training  program,  or  care  should  be  taken 
to  ensure  that  these  skills  are  acquired  else¬ 
where.  The  importance  of  this  statement  is 
underscored  by  considerable  anecdotal  re¬ 
ports  by  the  military  that  their  technicians 
do  not  know  how  to  use  test  equipment. 


It  was  also  noted  in  the  introduction  that 
the  appropriateness  of  any  strategy  depends 
greatly  upon  performance  criteria  and  other 
contingencies.  This  is  an  issue  that  is  not 
dealt  with  explicitly  to  any  great  extent  in 
training  programs,  possibly  because  the  de¬ 
mands  on  graduates  of  a  training  program 
may  differ  widely  across  individuals.  For  ex¬ 
ample,  the  aircraft  mechanic  who  goes  to 
work  for  a  major  airline  may  be  expected  to 
minimize  down  time;  the  mechanic  at  a 
small  private  airport  might  be  more  con¬ 
cerned  with  the  cost  of  repair.  Similar  differ¬ 
ences  may  be  associated  with  line  versus 
depot  maintenance. 

The  performance  criteria  of  a  particular 
job  may  also  be  expected  to  change,  de¬ 
pending  on  circumstances.  For  example,  the 
demands  on  military  maintenance  personnel 
might  be  expected  to  change  dramatically  in 
the  event  of  a  war.  Thus,  it  may  be  worth¬ 
while  to  investigate  the  human's  ability  to 
change  his  or  her  strategy  dependent  upon 
conditions,  and  perhaps  to  examine  alterna¬ 
tive  ways  of  training  troubleshooters  to  take 
such  contingencies  into  consideration. 

Although  it  is  not  entirely  feasible  to  offer 
specific  advice  as  to  what  strategies,  if  any, 
should  be  explicitly  taught  to  trouble¬ 
shooters,  some  consistent  themes  do  emerge 
as  a  result  of  this  examination  of  the  research 
results  available.  Accordingly,  we  will  at¬ 
tempt  to  be  as  definitive  as  possible  with  re¬ 
gard  to  what  is  known  about  trouble¬ 
shooting.  In  most  cases,  the  following  state¬ 
ments  are  based  upon  the  empirical  evidence 
reported  here;  exceptions  to  this  will  be 
noted. 

It  may  be  stated  with  a  fair  amount  of  cer¬ 
tainty  that  humans  are  not  good  at  making 
probabilistic  judgments  and  generally  do  not 
make  very  good  use  of  such  information.  This 
was  found  in  studies  dealing  specifically  with 
troubleshooting  and  is  consistent  with  a  siz¬ 
able  body  of  literature  in  the  area  of  human 
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judgment  and  decision  making.  Subject- 
matter  experts  exhibit  the  same  kinds  of 
error  in  probabilistic  judgments  as  do 
novices,  and  merely  informing  people  that 
their  judgments  are  subject  to  such  biases 
apparently  has  little  impact.  The  only  people 
who  have  been  found  to  make  relatively  bias- 
free  judgments  of  probabilities  are  those  who 
are  experts  at  making  such  judgments  (e.g., 
weather  forecasters). 

The  lesson  to  be  learned  from  this  is  that 
although  troubleshooters  may  base  their  de¬ 
cisions  on  qualitative  perceptions  of  the  like¬ 
lihood  of  certain  events,  they  should  not  be 
expected  to  make  precise  judgments  about 
probabilities  and  should  not  be  expected  to 
optimize  performance  based  on  such  proba¬ 
bilities.  If  it  can  be  determined  that  it  is  abso¬ 
lutely  necessary  to  take  probabilistic  infor¬ 
mation  into  account,  then  provision  of  some 
kind  of  aid  should  be  considered.  Otherwise, 
reliance  on  this  type  of  information  should  be 
avoided. 

Human  troubleshooting  performance  has 
been  found  to  degrade  as  systems  increase  in 
size  or  complexity,  and  if  time  constraints 
are  imposed.  If  the  system  involved  is  large 
or  complex,  and/or  if  time  is  limited,  then 
one  or  more  of  the  following  approaches 
should  be  considered:  (1)  require  a  relatively 
simple  strategy;  (2)  provide  more  instruction 
and  practice  in  applying  the  instruction;  (3) 
select  troubleshooters  with  higher  ability;  (4) 
provide  some  kind  of  performance  aid,  such 
as  "bookkeeping”  or  procedures.  There  are 
obviously  trade-offs  involved  in  adopting  any 
of  these  approaches,  and  the  choice  should 
depend  upon  such  contingencies  as  the  popu¬ 
lation  from  which  troubleshooters  may  be 
drawn,  the  resources  that  may  be  allocated  to 
each  of  these,  and  the  extent  to  which  each 
approach  may  be  expected  to  lead  to  accept¬ 
able  performance. 

Evidence  suggests  that  as  instruction  to 
troubleshooters  is  made  more  explicit,  their 


behavior  will  probably  be  affected  in  the 
manner  desired.  Generally,  if  the  trouble¬ 
shooter  is  left  to  discover  the  best  strategy  for 
a  given  situation,  this  discovery  will  not 
occur.  This  interpretation  is  consistent  with 
virtually  all  of  the  research  reviewed,  al¬ 
though  perhaps  the  best  illustration  of  this  is 
the  overall  ineffectiveness  of  theoretical  in¬ 
struction  in  enabling  persons  to  be  good  trou¬ 
bleshooters.  Granted,  if  the  trainees  involved 
have  high  ability  and/or  are  highly  moti¬ 
vated,  they  may  be  able  to  develop  an  appro¬ 
priate  strategy  given  such  instruction.  In  gen¬ 
eral,  however,  if  the  leap  from  instruction  to 
action  is  too  great,  that  leap  will  not  be  made 
successfully. 

From  this  perspective,  the  most  effective 
means  of  ensuring  that  the  troubleshooter 
will  successfully  employ  an  appropriate 
strategy  is  to  proceduralize  the  task.  Positive 
effects  with  procedures  were  noted  in  each 
study  in  which  proceduralization  was  used, 
even  when  troubleshooters  had  limited  expe¬ 
rience  or  moderate  ability.  However,  we  re¬ 
alize  that  there  are  important  limitations  to 
this  approach.  Procedures  are  not  easily  de¬ 
veloped,  and  they  are  highly  system-specific. 
In  hard-copy  form,  they  may  be  cumbersome 
and  space  consuming,  and  total  reliance  on  a 
procedures  manual  implies  that  little  could 
be  done  if  no  manual  were  on  hand.  There¬ 
fore,  explicit  instruction  in  some  algorithm 
or  in  diagnostic  heuristics,  either  of  which 
would  have  greater  generality  and  would  re¬ 
sult  in  less  dependence  on  a  manual,  seems  to 
be  a  desirable  alternative  to  proceduraliza¬ 
tion. 

It  is  possible  to  imagine  that  some  in¬ 
structors  would  prefer  to  have  trouble¬ 
shooters  rely  on  their  knowledge  of  the 
system  rather  than  on  an  algorithm  or  on 
rules.  Even  if  this  were  to  be  the  focus  of 
training,  a  case  may  be  made  for  explicit  ac¬ 
tion-related  instruction.  In  studies  in  which 
troubleshooters  were  given  specific  guidance 
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in  how  to  apply  their  knowledge  (e.g.,  or- 
ganize  information  into  chunks  or  generate 
hypotheses  and  plan  an  appropriate  course  of 
action  prior  to  beginning  work),  the  provi¬ 
sion  of  this  guidance  led  to  better  perfor¬ 
mance. 

This  call  for  explicitness  is  contrary  to  the 
approach  in  which  the  trainee  is  not  told 
what  to  do  but  rather  is  placed  in  a  situation 
that  is  intended  to  force  the  use  of  system-rel¬ 
evant  knowledge  in  order  to  determine  a 
course  of  action.  However,  limited  support 
for  this  latter  approach  can  be  found  in  re¬ 
search  indicating  that  forcing  people  (as  op¬ 
posed  to  providing  them  with  the  opportu¬ 
nity)  to  use  the  information  they  have  can 
lead  to  improved  performance.  Most  of  the 
evidence  is  directly  relevant  to  the  use  of  an 
algorithm  or  heuristics  rather  than  theoret¬ 
ical  knowledge,  but  the  results  of  one  study 
and  aspects  of  another  do  suggest  that  this 
may  also  be  the  case  with  knowledge  of  the 
system. 

Perhaps  the  current  state  of  knowledge 
may  be  summarized  as  follows.  Either  trou¬ 
bleshooters  should  be  explicitly  instructed  in 
how  to  approach  problems  or  they  should  be 
forced  to  use  their  knowledge  of  the  system 
explicitly  in  deciding  what  to  do.  Unfortu¬ 
nately,  a  definitive  statement  as  to  which  of 
these  is  the  better  approach  cannot  be  made 
because  of  a  lack  of  data.  The  latter  has  not 
been  evaluated  in  a  transfer  study,  and  the 
two  approaches  have  not  been  compared 
with  each  other  experimentally. 

It  seems  feasible  that  a  combination  of 
these  approaches,  taking  advantage  of  the 
possible  benefits  of  each,  could  be  the  most 
effective  means  of  teaching  troubleshooting. 
The  use  of  an  algorithm  or  heuristics  could 
impose  less  cognitive  load  on  the  trouble¬ 
shooter  and  could  be  successful  even  when 
knowledge  of  the  system  is  limited.  On  the 
other  hand,  a  knowledge  of  the  functioning  of 
the  system  could  enable  the  troubleshooter  to 


focus  on  a  portion  of  the  total  system  and,  in 
effect,  reduce  the  size  and  complexity  of  the 
system  under  consideration.  This  in  turn 
could  make  it  possible  for  the  troubleshooter 
to  use  an  algorithm  such  as  a  half-split  ap¬ 
proach  more  effectively. 

Admittedly,  this  is  conjecture  and  is  not 
based  upon  empirical  evidence,  but  it  does 
appear  to  be  reasonable.  The  extent  to  which 
requiring  people  to  use  their  theoretical 
knowledge  can  enable  them  to  troubleshoot 
more  effectively  is  largely  unknown  and 
should  be  evaluated  experimentally.  Further 
evidence  is  also  needed  to  determine  the  ap¬ 
propriate  degree  of  emphasis  on  algorithmic 
and  theoretical  approaches. 

As  a  final  observation,  we  have  noted  a 
characteristic  of  the  literature  reviewed.  Sev¬ 
eral  of  the  studies  summarized  here  were 
conducted  some  time  ago  and  typically  used 
low-tech  approaches  to  experimentation, 
such  as  paper-and-pencil  simulations.  How¬ 
ever,  the  goal  of  these  experiments  was  to 
learn  something  about  human  trouble¬ 
shooting  behavior,  and  transfer  to  actual 
equipment  was  often  investigated.  Generally, 
the  data  generated  by  these  studies  are  useful 
today. 

In  contrast,  several  more  recent  reports 
have  discussed  high-tech  approaches,  with 
sophisticated  computer-based  simulations, 
elaborate  graphics,  “expert”  models,  etc.  Yet, 
accompanying  this  trend  toward  greater  use 
of  technology,  there  seems  to  have  been  a 
shift  in  emphasis  that  avoided,  either  pur¬ 
posefully  or  incidentally,  an  evaluation  of 
transfer  of  training.  All  too  often,  recent  “ex¬ 
periments”  have  been  demonstrations  rather 
than  evaluations.  In  light  of  the  large  number 
of  questions  remaining,  this  is  an  unfortunate 
trend.  Verification  of  the  feasibility  of  an  idea 
is  a  necessary  step,  but  it  should  not  be 
viewed  as  the  final  step.  Research  should  re¬ 
tain  as  its  primary  goal  the  evaluation  of 
transfer  to  real-life  situations. 


528 — October,  1985 


HUMAN  FACTORS 


ACKNOWLEDGMENTS 

This  work  was  supported  by  the  Navy  Personnel  Re¬ 
search  and  Development  Center  under  Contract  No 
N66001-83-R-0340. 


REFERENCES 

Baldwin,  R.  D.  (1978,  April).  Training  the  electronics  main¬ 
tenance  technician  (HumRRO  Prof.  Paper  7-78).  Alex¬ 
andria,  VA:  Human  Resources  Research  Organization. 

Bond,  N.A.,  Jr.,  and  Rigney,  J.  W.  (1966).  Bayesian 
aspects  of  trouble  shooting  behavior.  Human  Factors 
8,  377-383. 

Brigham,  F.,  and  Laios,  L.  (1975).  Operator  performance 
in  the  control  of  a  laboratory  process  plant.  Ergo¬ 
nomics,  18,  53-66. 

Brooke,  J.  B.,  Cooke,  J.  F.,  and  Duncan,  K.  D.  (1983).  Ef¬ 
fects  of  computer  aiding  and  pretraining  on  fault  loca¬ 
tion.  Ergonomics,  26,  669-686. 

Brooke,  J.  B.,  and  Duncan,  K.  D.  (1981).  Effects  of  system 
display  format  on  performance  in  a  fault  location  task. 
Ergonomics,  24,  175-189. 

Brooke,  J.  B.,  Duncan,  K.  D.,  and  Cooper,  C.  (1980).  Inter¬ 
active  instruction  in  solving  fault-finding  problems — 
An  experimental  study.  International  Journal  of  Man- 
Machine  Studies,  12,  217-227. 

Brooke,  J.  B.,  Duncan,  K.  D.,  and  Marshall,  E.  C.  (1978). 
Interactive  instruction  in  solving  fault-finding 
problems.  International  Journal  of  Man-Machine 
Studies,  10,  603-611. 

Brown,  J.  S.,  Rubenstein,  R.,  and  Burton,  R.  (1976,  Oc¬ 
tober).  Reactive  learning  environment  for  computer  as¬ 
sisted  electronics  instruction  (Tech.  Report  AFHRL-TR- 
76-68).  Cambridge,  MA:  Bolt  Beranek  and  Newman. 

Cicchinelli,  L.  F.,  Keller,  R,  A.,  and  Harmon,  K.  R.  (1984, 
February).  Training  capabilities  test  of  Electronics 
Equipment  Maintenance  Trainer  (EEMT):  Findings  and 
conclusions  (Draft  report).  Denver,  CO:  University  of 
Denver,  Denver  Research  Institute. 

Crooks,  W.H.,  Kuppin,  M.  A.,  and  Freedv,  A.  (1977,  Jan¬ 
uary).  Application  of  adaptive  decision  aiding  systems  to 
computer-assisted  instruction:  Adaptive  Computerized 
Training  System  (ACTS)  (Tech.  Report  PATR-1028-77- 
1).  Woodland  Hills,  CA:  Perccptronics. 

Crossman,  E.  R.  F.  W.,  and  Cooke,  J.  E.  (1974).  Manual 
control  of  slow-response  systems.  In  E.  Edwards  and 
F  P.  Lees  (Eds.),  The  human  operator  in  process  control 
(pp.  51-66).  London:  Taylor  and  Francis. 

Dale,  H.C.  A.  (1958).  Fault-finding  in  electronic  equip¬ 
ment.  Ergonomics ,  1 ,  356-385. 

Demaree,  R.  G.,  Crowder,  N.  A.,  and  Morrison,  E.  J  (1955, 
March).  Proficiency  of  Q-24  radar  mechanics:  Summary 
of  findings  (Tech.  Memorandum  ASPRL  TM-55-6). 
Lowry  Air  Force  Base,  CO:  ASPRL. 

Duncan,  K.  D.  (1971).  Long-term  retention  and  transfer  of 
an  industrial  search  skill.  British  Journal  of  Psychology 
62,  439-448. 

Duncan,  K.  D.,  and  Shepherd,  A.  (1975).  A  simulator  and 
training  technique  for  diagnosing  plant  failures  from 
control  panels.  Ergonomics,  18,  627-641. 

Elliott,  T.  K.  (1965,  November).  Effect  of  format  and  detail 
of  job  performance  aids  in  performing  simulated  trouble¬ 
shooting  tasks  (Tech.  Report  AMRL-TR-65-1 54).  Va¬ 
lencia,  PA:  Applied  Science  Associates. 


Elliott,  T.  K.  (1966,  December).  A  comparison  of  three 
methods  for  presenting  procedural  troubleshooting  infor¬ 
mation  (Tech.  Report  AMRL-TR-66-191).  Valencia,  PA: 
Applied  Science  Associates. 

Elliott,  T.  K.  (1967,  November).  The  effect  of  electronic 
aptitude  on  performance  of  proceduralized  trouble¬ 
shooting  tasks  (Tech.  Report  AMRL-TR-67-154).  Va¬ 
lencia,  PA;  Applied  Science  Associates. 

Elliott,  T.  K.,  and  Joyce,  R.  P.  (1968,  November).  An  exper¬ 
imental  comparison  of  procedural  and  conventional  elec¬ 
tronic  troubleshooting  (Tech.  Report  AFHRL-TR-68-1). 
Valencia,  PA:  Applied  Science  Associates. 

Elliott,  T.  K.,  and  Joyce,  R  P  (1971).  An  experimental 
evaluation  of  a  method  for  simplifying  electronic 
maintenance.  Human  Factors,  13,  217-227. 

Fattu,  N  A.,  and  Mech,  E.  V.  (1953).  The  effect  of  set  on 
performance  in  a  "trouble  shooting"  situation.  Journal 
of  Applied  Psychology ,  37,  214-217. 

Federico,  P-A.  (1982).  Individual  differences  in  cognitive 
characteristics  and  computer-managed  masterv 
learning.  Journal  of  Computer-Based  Instruction,  9, 
10-18. 

Federico,  P-A.  (1983).  Changes  in  the  cognitive  compo¬ 
nents  of  achievement  as  students  proceed  through 
computer-managed  instruction.  Journal  of  Computer- 
Based  Instruction,  9,  156-168. 

Federico,  P-A  ,  and  Landis,  D.  B.  (1979,  August).  Predicting 
student  performance  in  a  computer-managed  course 
using  measures  of  cognitive  styles,  abilities,  and  apti¬ 
tudes  (Tech.  Report  NPRDC  TR  79-30).  San  Diego,  CA: 
Navy  Personnel  Research  and  Development  Center. 

Federico,  P-A.,  and  Landis,  D.  B.  (1980,  April).  Relation¬ 
ships  among  selected  measures  of  cognitive  styles,  abili¬ 
ties,  and  aptitudes  (Tech.  Report  NPRDC  TR  80-23). 
San  Diego,  CA:  Navy  Personnel  Research  and  Develop¬ 
ment  Center. 

Foley,  J  P.,  Jr.  (1977,  December).  Performance  measure¬ 
ment  of  maintenance  (Tech.  Report  AFHRL-TR-77-76). 
Wright-Patterson  Air  Force  Base,  OH:  Air  Force 
Human  Resources  Laboratory. 

Gettys,  C.  F.,  Manning,  C.,  Mehle,  T.,  and  Fisher,  S.  (1980, 
October).  Hypothesis  generation:  A  final  report  of  three 
years  of  research  (Tech.  Report  15-10-80).  Norman,  OK: 
University  of  Oklahoma,  Decision  Processes  Labora¬ 
tory. 

Glaser,  R.,  and  Phillips,  J.  C.  (1954,  August).  An  analysis  of 
proficiency  for  guided  missile  personnel:  / II.  Patterns  of 
troubleshooting  behavior  (Tech.  Bulletin  55-16).  Wash¬ 
ington,  DC:  American  Institute  for  Research. 

Glass,  A.  A.  (1967).  Problem-solving  techniques  and  trou¬ 
bleshooting  simulators  in  training  electronic  re¬ 
pairmen.  Dissertation  Abstracts  International,  28, 
1717B-1718B.  (University  Microfilms  No.  67-12,  252) 

Goldbeck,  R.  A.  Bernstein  B  B.,  Hillix,  W.  A.,  and  Marx, 
M.  H.  (1957).  Application  of  the  half-split  technique  to 
problem-solving  tasks.  Journal  of  Experimental  Psy¬ 
chology,  53,  330-338. 

Henneman,  R.  L.,  and  Rouse,  W.  B.  (1984).  Measures  of 
human  problem  solving  performance  in  fault  diag¬ 
nosis  tasks.  IEEE  Transactions  on  Systems,  Man,  and 
Cybernetics,  SMC- 14,  99-112. 

Highland,  R.  W  ,  Newman,  S.  E.,  and  Waller,  H.  S.  (1956). 
A  descriptive  study  of  electronic  trouble  shooting.  In 
Air  Force  human  engineering,  personnel,  and  training  re¬ 
search  (Tech.  Report  56-8).  Baltimore,  MD:  Air  Re¬ 
search  and  Development  Command. 


TROUBLE  SHOOTING 


October,  1985 — 529 


Hopf-Weichel,  R.,  Purcell,  D,,  Freedv,  A.,  and  Lucaccini,  L. 
(1979,  November).  Adaptive  decision  aiding  in  com¬ 
puter-assisted  instruction:  Adaptive  Computerized 
Training  System  {ACTS)  (Tech.  Report  PDATR- 1076-79- 
1 1).  Woodland  Hills,  CA:  Perceptronics. 

Hunt,  R.  M.  (1981 ,  September).  Human  pattern,  recognition 
and  information  seeking  in  simulated  fault  diagnosis 
tasks  (Tech.  Report  T-110).  Urbana,  IL:  University  of 
Illinois  at  Urbana-Champaign,  Coordinated  Science 
Laboratory. 

Hunt,  R.  M.,  Henneman,  R.  L.,  and  Rouse,  W.  B.  (1981). 
Characterizing  the  development  of  human  expertise  in 
simulated  fault  diagnosis  tasks.  In  Proceedings  of  the 
International  Conference  on  Cybernetics  and  Society 
(pp.  369-374).  Atlanta,  GA:  IEEE  Systems,  Man,  and 
Cybernetics  Society. 

Hunt,  R.  M.,  and  Rouse,  W.  (1981).  Problem-solving  skills 
of  maintenance  trainees  in  diagnosing  faults  in  simu¬ 
lated  powerplants.  Human  Factors,  23,  317-328. 

Johnson,  W.  B.,  and  Fath,  J.  L.  (1984,  Julv).  Implementa¬ 
tion  of  a  mixed-fidelity  approach  to  technical  training 
(Tech.  Report  82-2).  Norcross,  GA:  Search  Technology. 

Johnson,  W.  B.,  and  Rouse,  W.  B.  (1982a).  Analysis  and 
classification  of  human  errors  in  troubleshooting  live 
aircraft  powerplants.  IEEE  Transactions  on  Systems, 
Man,  and  Cybernetics,  SMC-12,  389-393. 

Johnson,  W.  B.,  and  Rouse,  W.  B,  (1982b).  Training  main¬ 
tenance  technicians  for  troubleshooting:  Two  experi¬ 
ments  with  computer  simulations.  Human  Factors,  24, 
271-276. 

Kragt,  H.,  and  Landewecrd,  J.  A.  (1974).  Mental  skills  in 
process  control.  In  E.  Edwards  and  F.  P  Lees  (Eds.), 
The  human  operator  in  process  control  (pp.  135-145). 
London:  Taylor  and  Francis. 

Levine,  J  M.,  Brahlek,  R.  E.,  Eisner,  E.  J.,  and  Fleishman, 
E.  A.  (1979,  March).  Trainability  of  abilities:  Training 
and  transfer  of  abilities  related  to  electronic  fault-finding 
(Tech.  Report  ARRO-3010-TR2).  Washington,  DC:  Ad¬ 
vanced  Research  Resources  Organization. 

Levine,  J.M.,  Schulman,  D.  Brahlek,  R.E.,  and 
Fleishman,  E.  A.  (1980,  April).  Trainability  of  abilities : 
Training  and  transfer  of  spatial  visualization  (Tech.  Re¬ 
port  ARRO-3010-TR3).  Washington,  DC:  Advanced  Re¬ 
search  Resources  Organization. 

Mallory,  W.J.,  and  Elliott,  T.  K.  (1978,  December).  Mea¬ 
suring  troubleshooting  skills  using  hardware-free  simula¬ 
tion  (Tech.  Report  AFHRL-TR-78-47).  Valencia,  PA: 
Applied  Science  Associates. 

Marshall,  E.  C.,  Duncan,  K.  D.,  and  Baker,  S.  M.  (1981). 
The  role  of  withheld  information  in  the  training  of 
process  plant  fault  diagnosis.  Ergonomics,  24,  71 1-724. 

Marshall,  E.  C.,  Scanlon,  K.  E.,  Shepherd,  A.,  and  Duncan, 
K.  D.  (1981,  February).  Panel  diagnosis  training  for 
major-hazard  continuous-process  installations.  The 
Chemical  Engineer,  No.  365,  pp.  66-69. 

Mayer,  R.  E.,  and  Greeno,  J.G.  (1972).  Structural  differ¬ 
ences  between  learning  outcomes  produced  by  dif¬ 
ferent  instructional  methods.  Journal  of  Educational 
Psychology ,  63,  165-173. 

Mayer,  R.  E.,  Stiehl,  C.,  and  Greeno,  J.  G.  (1975).  Acquisi¬ 
tion  of  understanding  and  skill  in  relation  to  subjects' 
preparation  and  meaningfulness  of  instruction. 
Journal  of  Educational  Psychology,  67,  331-350. 

McDonald,  L.  B„  Waldrop,  G.  P.,  and  White,  V.  T.  (1983, 
December).  Analysis  of  fidelity  requirements  for 
electronics  equipment  maintenance  (Tech.  Report 


NAVTRAEQUIPCEN  8 l-C-0065-1).  Orlando,  FL:  Naval 
Training  Equipment  Center. 

Mehle,  T.  (1980,  February).  Hypothesis  generation  in  an  au¬ 
tomobile  malfunction  inference  task  (Tech.  Report  TR 
25-2-80).  Norman,  OK:  University  of  Oklahoma,  Deci¬ 
sion  Processes  Laboratory. 

Miller,  E.  E.  (1975,  July).  Instructional  strategies  using  low- 
cost  simulation  for  electronic  maintenance  (Tech.  Re¬ 
port  HumRRO-FR-WD(TX)-75-20).  Alexandria,  VA 
Human  Resources  Research  Organization. 

Mills,  R.  G.  (1971).  Probability  processing  and  diagnostic 
search.  20  alternatives,  500  trials.  Psychonomic 
Science,  24,  289-292. 

Moore,  J  V.,  Saltz,  E.,  and  Hoehn,  A.  J.  (1955,  September). 
Improving  equipment  maintenance  by  means  of  a  pre¬ 
planning  technique  (Tech.  Report  AFPTRC-TN-55-26). 
Lackland  Air  Force  Base,  TX:  Air  Force  Personnel 
Training  Research  Center. 

Morris,  N.  M.,  and  Rouse,  W.  B.  (in  press).  The  effects  of 
type  of  knowledge  upon  human  problem  solving  in  a 
process  control  task.  IEEE  Transactions  on  Systems, 
Man,  and  Cybernetics. 

Potter,  N.  R  ,  and  Thomas,  D.  L.  (1976,  September).  Evalu¬ 
ation  of  three  types  of  technical  data  for  troubleshooting : 
Results  and  project  summary  (Tech.  Report  AFHRL-TR- 
74  (I)).  Dayton,  OH:  Systems  Research  Laboratories. 

Rigney,  J.  W.,  and  Hoffman,  L.  S.  (1961,  June).  An  experi¬ 
mental  study  of  factors  influencing  trouble-shooting  dif¬ 
ficulty  (Tech.  Report  33).  Los  Angeles:  University  of 
Southern  California,  Electronics  Personnel  Research 
Group. 

Rigney,  J.  W.,  Towne,  D  M.,  King,  C.  A.,  and  Moran,  P.  J . 
(1978,  October).  Field  evaluation  of  the  Generalized 
Maintenance  Trainer-Simulator:  I.  Fleet  communica¬ 
tions  system  (Tech.  Report  89).  Los  Angeles:  University 
of  Southern  California,  Behavioral  Technology  Labo¬ 
ratories. 

Rigney,  J.W.,  Towne,  D  M.,  Moran.  P.  J.,  and  Mishler, 
R.  A.  (1980,  July).  Field  evaluation  of  the  Generalized 
Maintenance  Trainer-Simulator:  II.  ANISPA-66  radar  re¬ 
peater  (Tech.  Report  80-30).  San  Diego,  CA:  Navy  Per¬ 
sonnel  Research  and  Development  Center. 

Rose,  A.  M.,  Fingerman,  P.  W.,  Wheaton,  G  R.P  Eisner,  E 
and  Kramer,  G.  (1974,  August).  Methods  for  predicting 
job-ability  requirements:  II.  Ability  requirements  as  a 
function  of  changes  in  the  characteristics  of  an  elec¬ 
tronic  fault-finding  task  (Tech.  Report  R74-6).  Wash¬ 
ington,  DC:  American  Institute  for  Research. 

Rouse,  S.  H.,  and  Rouse,  W  B.  (1982).  Cognitive  style  as  a 
correlate  of  human  problem  solving  performance  in 
fault  diagnosis  tasks.  IEEE  Transactions  on  Systems, 
Man,  and  Cybernetics,  SMC-12,  649-652. 

Rouse,  W.  B  (1978a).  Human  problem  solving  perfor¬ 
mance  in  a  fault  diagnosis  task.  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics,  SMC-8,  258-271. 

Rouse,  W.  B.  (1978b).  A  model  of  human  decision  making 
in  a  fault  diagnosis  task.  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics,  SMC-8,  357-361 . 

Rouse,  W.  B.  (1979a).  A  model  of  human  decision  making 
in  fault  diagnosis  tasks  that  include  feedback  and  re¬ 
dundancy.  IEEE  Transactions  on  Systems,  Man,  and 
Cybernetics,  SMC-9,  237-241. 

Rouse,  W.  B.  (1979b).  Problem  solving  performance  of  first 
semester  maintenance  trainees  in  two  fault  diagnosis 
tasks.  Human  Factors,  21,  611-618. 


530— October,  1985 


HUMAN  FACTORS 


Rouse,  W.B.  (1979c).  Problem  solving  performance  of 
maintenance  trainees  in  a  fault  diagnosis  task.  Human 
Factors,  21,  195-203. 

Rouse,  W.  B.,  and  Hunt,  R.  M.  (1984).  Human  problem 
solving  in  fault  diagnosis  tasks.  In  W.  B.  Rouse  (Ed.). 
Advances  in  man-machine  systems  research  (Vol.  1)  (pp. 
195-222).  Greenwich,  CT:  JAI  Press. 

Rouse,  W.  B.f  and  Rouse,  S.  H.  (1979).  Measures  of  com¬ 
plexity  of  fault  diagnosis  tasks.  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics,  SMC-9,  720-727. 

Rouse,  W.  B,  Rouse,  S.  H.,  and  Pellegrino,  S.  J.  (1980).  A 
rule-based  model  of  human  problem  solving  perfor¬ 
mance  in  fault  diagnosis  tasks.  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics,  SMC-10,  366-376. 

Saltz,  E.,  and  Moore,  J.  V.  (1953,  February).  A  preliminary 
investigation  of  trouble  shooting  (Tech.  Report  53-2). 
Lackland  Air  Force  Base,  TX:  Human  Resources  Re¬ 
search  Center. 

Saupe,  J.  L.  (1954).  Troubleshooting  electronic  equip¬ 
ment'  An  empirical  approach  to  the  identification  of 
certain  requirements  of  a  maintenance  occupation. 
(Doctoral  dissertation,  University  of  Illinois,  1954.) 
Dissertation  Abstracts  International,  14,  1966. 

Schorgmayer,  H.,  and  Swanson,  R.  A.  (1975,  August).  In¬ 
dustrial  training  research :  II.  The  effect  of  alteniative 
training  methods  on  the  troubleshooting  performance  of 
maintenance  technicians .  Bowling  Green,  OH:  Bowling 
Green  State  Universitv. 

Shepherd,  A.,  Marshall,  E.  C.,  Turner,  A.,  and  Duncan, 
K.  D,  (1977).  Diagnosis  of  plant  failures  from  a  control 
panel:  A  comparison  of  three  training  methods.  Ergo¬ 
nomics,  20,  347-361 . 

Shriver,  E.  L.,  and  Foley,  J.  P.,  Jr.  (1974,  November).  Eval¬ 
uating  maintenance  performance:  The  development  of 
graphic  symbolic  substitutes  for  criterion  referenced  job 
task  performance  tests  for  electronic  maintenance  (Tech. 
Report  AFHRL-TR-74-57).  Wright-Patterson  Air  Force 
Base,  OH:  Air  Force  Human  Resources  Laboratory. 

Smillie,  R.  J.,  and  Porta,  M.  M.  (1981,  October).  Compar¬ 
ison  of  state  tables  and  ordnance  publications  for  trou¬ 
bleshooting  digital  equipment  (Tech.  Report  NPRDC 
TR  82-7).  Fullerton,  CA;  Hughes  Aircraft  Corporation 

Standlee,  L.,  and  Bilinski,  C.  (1966).  Note  on  simulated  vs. 
actual  electronic  troubleshooting  performance.  Percep¬ 
tual  and  Motor  Skills,  23,  532. 

Steinemann,  J.  H.  (1966,  July).  Comparison  of  performance 


on  analogous  simulated  and  actual  troubleshooting  tasks 
(Res.  Memorandum  SRM  67-1).  San  Diego,  CA  U.S. 
Navy  Training  Research  Laboratory. 

Steinemann,  J.  H.,  Harrigan,  R.  J.,  and  Van  Matre,  N.  H. 
(1967,  October).  A  performance-oriented  electronics 
training  program:  IV:  Fleet  follow-up  evaluation  of  grad¬ 
uates  of  all  classes  (Res.  Report  SRR  68-10).  San  Diego, 
CA:  U.S.  Naval  Personnel  Research  Activity. 

Stolurow,  L.  M.,  Bergum,  B  ,  Hodgson,  T.,  and  Silva,  J. 
(1955).  The  efficient  course  of  action  in  'trouble 
shooting"  as  a  joint  function  of  probability  and  cost. 
Educational  and  Psychological  Measurement  15 
462-477. 

Towne,  D.  M.,  Munro,  A.,  Johnson,  M.  C.,  and  Lahey,  G.  F. 
(1983,  August).  Generalized  maintenance  trainer  simu¬ 
lator:  Test  and  evaluation  in  the  laboratory  environment 
(Tech  Report  NPRDC  TR  83-28).  San  Diego,  CA:  Navy 
Personnel  Research  and  Development  Center. 

„Van  Matre,  N.H.,  and  Steinemann,  J.  H.  (1966,  De¬ 
cember).  A  performance-oriented  electronics  technician 
training  program:  II.  Initial  fleet  follow-up  evaluation  of 
graduates  (Tech.  Bulletin  STB  67-15).  San  Diego,  CA: 
U.S.  Naval  Personnel  Research  Activity. 

Vineberg,  R.  A.  (1955,  August).  A  performance  test  for  the 
AAFCS  M33  radar  mechanic  and  observations  on 
troubleshooting  behavior.  Paper  for  Symposium  on 
Electronics  Maintenance,  Office  of  the  Assistant  Secre¬ 
tary  of  Defense  Research,  and  Development,  Advisory 
Pane!  on  Personnel  and  Training  Research.  Reprinted 
in  Collected  papers  prepared  under  work  unit  RADAR 
(HumRRO  Prof.  Paper  20-68).  Alexandria,  VA:  George 
Washington  University,  Human  Resources  Research 
Office,  June  1968. 

Williams,  W.  L.,  Jr.,  and  Whitmore,  P.  G.,  Jr.  (1959).  The 
development  and  use  of  a  performance  test  as  a  basis  for 
comparing  technicians  with  and  without  field  expeii- 
ence:  The  Nike  Ajax  maintenance  technician  (Tech.  Re¬ 
port  52).  Washington,  DC:  George  Washington  Univer¬ 
sity,  Human  Resources  Research  Office. 

Wohl,  J  G.  (1961).  Research  data  on  maintainability. 
IEEE  Transactions  of  the  Professional  Group  on  Human 
Factors  in  Electronics,  HFE-2 ,  112-113. 

Wohl,  J.  G.  (1982).  Maintainabili  ty  prediction  revisited: 
Diagnostic  behavior,  system  complexity,  and  repair 
time.  IEEE  Transactions  on  Systems,  Man,  and  Cyber¬ 
netics,  SMC- 12,  241-250. 


DISTRIBUTION  LIST 


Deputy  Under  Secretary  of  Defense  for  Research  and  Engineering  (Research  and 
Advanced  Technology) 

Military  Assistant  for  Training  and  Personnel  Technology  (OSUDS)  (R&AT) 

Assistant  Secretary  of  the  Navy  (Manpower  and  Reserve  Affairs)  (OASN)  (M&RA) 

Deputy  Assistant  Secretary  of  the  Navy  (Manpower) 

Chief  of  Naval  Operations  (OP-01B7)  (2),  (OP-11),  (OP-987H) 

Director  of  Navy  Laboratories  (ONT  0722) 

Commanding  Officer,  Service  School  Command  (Code  3200) 

Chief  of  Naval  Research  (Code  442PT) 

Chief  of  Naval  Education  and  Training  (Code  N-21) 

Director,  Naval  Education  and  Training  Program  Development  Center 
Chief  of  Naval  Technical  Training  (Code  N-31) 

Commanding  Officer,  Naval  Training  Equipment  Center  (Code  7) 

Commander,  Space  and  Naval  Warfare  System  Command  (SPAWAR-06/ASW-50) 
Commander,  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  Alexandria 
(PERI-POT— I) 

Commander,  Air  Force  Human  Resources  Laboratory,  Brooks  Air  Force  Base  (Scientific 
and  Technical  Information  Officer),  (TSRL/Technical  Library  FL  2870) 

Commander,  Air  Force  Human  Resources  Laboratory,  Lowery  Air  Force  Base  (AFHRL/ID) 
Superintendent,  Naval  Postgraduate  School 
Director  of  Research,  U.S.  Naval  Academy  (2) 

Center  for  Naval  Analyses 

Canadian  Forces  Personnel,  Applied  Research  Unit,  Canada 
DPSRSC,  National  Defense  Headquarters,  Canada 
Ministry  of  Defense,  Senior  Psychologist,  England  (Naval) 

D.  Dennison,  Army  Personnel  Research  Establishment,  Personnel  Psychological  Division, 
England  (2) 

Science  3  (RAF),  Lacon  House,  England 
1  Psychological  Research  Unit,  NBH  3-44,  Australia 

Directorate  of  Psychology  -  AF,  Department  of  Defense  (Air  for  CE),  Australia 
Navy  Psychology,  Australia  (2) 

Defense  Psychology  Unit,  Defense  HQ,  New  Zealand  (2) 

Dr.  Regan 

Defense  Technical  Information  Center  (DDAC)  (2) 


U224252 


tn 

> 

Z 

D 

rn 

O 

O 

O 

> 

(£> 

l\> 

U1 

ro 

o> 

CD 

8 


> 
< 
-< 
-o 
m 
;o 
tn 
O 
z 
z 

m 
r 
a 

ft  B 

52 

5  o 

> 

z 

o 


D 

m 

< 

m 

r 

O 

tj 

2 

m 

z 

H 


a 


m 

■ti 

> 

x 

H 

2 

m 

z 

H 

O 

"n 

H 

I 

m 

z 

> 

< 

■< 


Q3 
>  2 
3  > 

j  o 

n  " 
z  > 

H  Z 
O  D 

T1  H 

h  m 

I  pi 
m  w 


